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FOREWORD 


As  the  Army  embarks  on  its  transformation  to  the  future  force,  a  variety  of  changes  will  be 
required  to  meet  the  needs  of  the  Army  of  the  future.  One  such  need  is  innovation  in  the 
techniques  and  technologies  used  to  assemble  and  assign  personnel  to  specific  assignments  and 
teams.  In  response  to  this  need,  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI)  issued  a  Small  Business  Innovation  Research  (SBIR)  Phase  I  contract  entitled 
“Identifying  the  Interpersonal/Interaction  (I/I)  Knowledge,  Skills,  and  Attributes  for  the 
Objective  Force  Soldier.”  The  aim  of  the  contract  was  to  identify  the  specific  I/I  KSAs  that 
would  help  the  Soldiers  of  the  future  succeed  in  the  small  team-oriented  environment  that  is  part 
of  the  Army’s  vision  for  transformation  and  develop  innovative  concepts  for  assessing  these 
KSAs.  The  Phase  I  research  and  development  efforts  have  laid  the  groundwork  for  the 
development  of  a  two-stage  assessment  process  under  a  Phase  II  contract.  The  work  will  enable 
development  of  innovative  assessments  for  selecting  and  assigning  the  FF  Soldier  in  a  variety  of 
contexts. 

ARI’s  Selection  and  Assignment  Research  Unit  conducts  research,  studies,  and  analyses  of 
individual  difference  measures  (of  aptitudes,  motivations,  and  other  attributes)  related  to 
Soldiers’  job  performance.  The  primary  goal  is  to  improve  the  Army’s  selection  and 
classification,  promotion,  and  reassignment  of  enlisted  Soldiers  and  officers.  The  research 
presented  in  this  report  demonstrates  how  the  Small  Business  Innovation  Research  (SBIR) 
Program  can  support  these  objectives. 

SCOTT  E.  GRAHAM 

Acting  Technical  Director 
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Identifying  and  Assessing  Interaction  Knowledges,  Skills,  and  Attributes  for 
Objective  Force  Soldiers 

EXECUTIVE  SUMMARY 


Research  Requirement 

The  future  force  has  defined  the  U.S.  Army  as  it  is  expected  to  exist  within  the  next  30  years 
(U.S.  Army,  2001).  The  future  force  will  be  supported  by  Future  Combat  Systems  (FCSs)  that 
will  improve  the  speed,  maneuverability,  fighting  capacity,  and  survivability  of  the  Army’s 
combat  operations.  Transformation  to  the  future  force  obviously  requires  tremendous  advances 
in  technology  to  provide  the  FCSs  that  will  enable  the  future  force.  Just  as  importantly,  however, 
the  Army  recognizes  the  importance  of  its  Soldiers  in  accomplishing  the  transformation,  both  in 
terms  of  making  the  transition  and  working  effectively  within  the  new  systems.  This  transition 
will  require  future  force  Soldiers  to  possess  unprecedented  interpersonal  skills  in  order  to 
achieve  success  in  their  new  unit  structures.  The  goal  of  this  Small  Business  Innovation  Research 
Program  (SBIR)  Phase  I  effort  was  to  identify  the  interpersonal  skills  required  of  the  future  force 
Soldiers  and  identify  innovative  strategies  for  measuring  those  skills  in  future  Soldiers. 

Procedure 

The  project  team  conducted  a  literature  review  to  identify  existing  Interpersonal/Interaction 
knowledge,  skill  and  attribute  (I/I  KSA)  taxonomies  that  would  begin  to  identify  potential 
interpersonal  KSAs  that  may  be  applicable  to  the  future  force  environment.  In  addition  to 
reviewing  existing  taxonomies,  the  literature  review  examined  papers  outlining  concepts  for  the 
future  force  and  other  studies  aimed  specifically  at  identifying  characteristics  of  the  future  force 
Soldier.  The  results  of  the  literature  review  were  used  as  a  starting  point  from  which  to  develop 
the  final  taxonomy  of  I/I  KSAs  for  the  Phase  I  effort. 

The  project  team  also  identified  measurement  approaches  that  could  potentially  be  used  to  assess 
I/I  KSAs.  This  effort  included  a  survey  of  commercial  and  government-off-the-shelf  measures  of 
I/I  KSAs  to  see  if  any  existing  measures  would  be  applicable  to  the  current  effort.  This  review 
also  catalogued  a  set  of  measurement  techniques  that  could  be  implemented  to  measure  the  I/I 
KSAs.  From  this  list  of  existing  measures  and  measurement  techniques  we  evaluated  which 
techniques  or  measures  could  be  used  to  provide  a  creative  but  practical  assessment  of  the  I/I 
KSAs. 

Findings 

The  I/I  KSA  taxonomy  adopted  by  the  project  team  includes  eleven  core  KSAs:  (a)  Relating  to 
and  Supporting  Others,  (b)  Cultural  Tolerance,  (c)  Conflict  Management,  (d)  Dependability,  (e) 
Team  Orientation,  (f)  Adaptability/Flexibility,  (g)  Social  Perceptiveness,  (h)  Communicating 
Orally,  (i)  Written  Communication,  (j)  Leading  Others,  and  (k)  Cooperativeness  in  Problem 
Solving.  To  assess  these  KSAs,  the  project  team  developed  a  plan  for  a  2-stage  assessment 
process.  Stage  1  consists  of  knowledge  measures  of  the  I/I  KSAs  and  Stage  2  requires  examinees 
to  demonstrate  I/I  skills  in  a  more  realistic  setting.  The  stage  1  assessment  will  consist  of  a 
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situational  judgment  test  or  action  exam  along  with  an  existing  biographical  data  instrument.  In 
the  initial  version  of  the  assessment,  at  least  one  marker  personality  assessment  will  be  used  to 
help  evaluate  the  construct  validity  of  other  measures  in  the  assessment  package.  A  marker  test  is 
used  to  compare  examinees'  scores  on  the  previously  validated  (marker)  test  to  scores  on  the 
experimental  measure.  If  the  experimental  measure  works  as  it  is  supposed  to  (that  is,  if  it 
measures  the  characteristic(s)  well),  then  people  who  score  high  (low)  on  the  marker  test  should 
also  score  high  (low)  on  the  experimental  measure-that  is,  they  should  score  the  same  on  both 
measures.  Stage  2  will  consist  of  a  leaderless  group  discussion  and  a  structured  interview  that 
will  be  video  taped  for  later  scoring  by  a  trained  assessor.  The  cornerstone  of  the  2-stage 
measurement  approach  will  be  the  leveraging  of  emerging  technology  in  computer  graphics  and 
simulation  to  provide  the  assessment  materials  to  examinees  via  computer.  The  exams  will 
utilize  computer  simulation  and  exercise  development  templates  to  provide  stimuli  to  test  takers 
and  allow  for  easy  administration,  scoring,  and  adaptation  of  the  exercises  to  a  variety  of 
environments. 

Utilization  of  Findings 

These  findings  will  be  used  as  a  guiding  strategy  for  developing  a  prototype  of  the  2-stage  I/I 
KS  A  assessment  battery  under  a  Phase  II  contract. 
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IDENTIFYING  AND  ASSESSING  INTERACTION  KNOWLEDGES,  SKILLS  AND 
APTITUDES  FOR  OBJECTIVE  FORCE  SOLDIERS 

PHASE  I  FINAL  REPORT 

OVERVIEW 

In  2003,  Micro  Analysis  and  Design  (MA&D)  and  the  Human  Resources  Research  Organization 
(HumRRO)  were  awarded  a  Phase  I  Small  Business  Innovative  Research  (SBIR)  contract 
entitled  “Identifying  the  Interpersonal/Interaction  knowledge,  skills,  and  attributes  (KSAs)  for 
the  Objective  Force  Soldier.”  The  purpose  of  the  Phase  I  project  was  to  identify  the  KSAs  that 
will  be  required  of  the  future  force  (FF)  Soldier  specifically  related  to  interpersonal  interactions 
and  to  identify  or  develop  innovative  assessment  concepts  for  measuring  these  skills  for  use  in 
selection  and  assignment  applications.  Under  the  Phase  I  effort  the  MA&D  and  HumRRO  team 
conducted  a  literature  review  to  select  a  set  of  interpersonal/interaction  skills  for  the  FF  Soldier 
and  identified  existing  measures  and  measurement  methods  that  could  be  useful  in  assessing 
these  KSAs.  This  work  is  being  done  in  preparation  for  Phase  H,  in  which  we  will  develop 
assessment  tools  that  could  be  used  for  a  variety  of  purposes  (e.g.,  enlisted  classification,  training 
evaluation,  skills  certification,  selection  for  special  assignments). 

The  tasks  completed  in  Phase  I  include: 

(1)  Identify  the  interpersonal/interaction  (FI)  skills  for  FF  Soldiers 

(2)  Research  and  critique  interpersonal/interaction  KSA  measures/techniques 

(3)  Develop  a  KSA-by-method  measurement  plan 

(4)  Develop  innovative  assessment  concepts 

We  conducted  an  extensive  literature  review  to  identify  the  I/I  KSAs  likely  to  be  needed  by  FF 
Soldiers.  Next  we  developed  a  list  of  possible  assessment  measures  and  techniques.  This  list 
included  many  commercial  off-the-shelf  (COTS)  assessments  as  well  as  techniques  such  as 
situational  judgment  tests  and  leaderless  group  discussions.  We  then  created  a  KSA-by-method 
matrix  and  used  it  to  identify  which  measures  seemed  to  be  appropriate  measures  for  each  KSA. 
Finally,  we  discussed  how  to  adapt  current  MA&D  technology  and  products  to  use  them  in 
innovative  ways  to  measure  the  I/I  KSAs. 

The  remainder  of  this  report  will  document  the  findings  of  the  Phase  I  effort,  describe  the 
proposed  design  of  the  assessment  battery,  describe  the  final  assessment  concepts  chosen  to 
assess  the  interpersonal/interaction  (FI)  KSAs,  and  provide  a  brief  overview  of  the  tasks  to  be 
performed  in  the  Phase  II  effort. 
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Identifying  the  Interpersonal/Interaction  knowledge,  skills,  and  attributes  (KSAs)  for  the 

Objective  Force  Soldier 

Over  the  last  several  years,  the  U.S.  Army  Research  Institute  for  the  Behavioral  Sciences  (ARI) 
has  undertaken  several  studies  to  identify  the  KSAs  needed  by  Soldiers  in  the  FF.  The  NC021 
project  (Ford,  R.  Campbell,  J.  P.  Campbell,  Knapp,  &  Walker,  2000;  Knapp  et  al.,  2002) 
identified  measurement  methods  to  assess  a  broad  range  of  KSAs  across  two  eras  (2000-2010 
and  2010-2025).  The  Select21  project  (Sager  &  Russell,  2003)  is  developing  a  set  of  measures  to 
supplement  the  existing  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  subtests  in 
selecting  and  classifying  entry-level  Soldiers.  The  Select21  research  provided  a  starting  point  for 
identifying  the  KSAs  relevant  to  this  project.  The  current  work  differs  from  those  studies  by 
focusing  specifically  on  the  KSAs  required  for  effective  interpersonal  interactions.  Although  the 
prior  ARI  research  included  some  interpersonal  KSAs  (e.g..  Teamwork  Orientation,  Social 
Perceptiveness),  we  thought  a  targeted  investigation  of  I/I  KSAs  might  be  desirable  under  this 
effort.  So  we  supplemented  the  military  research  with  academic  research  literature  that  addressed 
I/I  KSAs.  The  research,  summarized  on  the  following  pages,  has  been  organized  according  to 
four  major  themes,  which  seemed  to  encompass  the  major  dimensions  of  interest: 

•  Building  and  maintaining  relationships 

•  Teamwork 

•  Team  Leadership 

•  Communication 

Building  and  Maintaining  Relationships 

In  any  working  environment,  it  is  useful  to  be  able  to  get  along  well  with  others.  This  ability 
could  be  even  more  important  in  the  Army,  where  one  not  only  works  with  the  same  people 
every  day,  but  also  eats  and  sleeps  with  them.  This  section  of  the  report  describes  the  literature 
related  to  the  KSAs  we  associated  with  Building  and  Maintaining  Relationships.  These  included 
relating  to  and  supporting  others,  agreeableness,  concern  for  Soldier  quality  of  life,  cultural 
tolerance,  and  conflict  management. 

A  number  of  research  efforts  have  described  a  trait  referred  to  as  interpersonal  competence  or 
general  interpersonal  skills  (Cannon-Bowers,  Tannenbaum,  Salas,  &  Volpe,  1995;  Ford  et  al., 
2000;  Knapp  et  al.,  2002;  Medsker  &  Campion,  1997;  Sager  &  Russell,  2003).  Medsker  and 
Campion  define  interpersonal  competence  as  the  “ability  to  maintain  healthy  working 
relationships  and  respond  to  others  with  respect  for  their  point  of  view”  (p.  475).  Ford  et  al. 
termed  a  similar  KSA  “Relating  to  and  supporting  others.”  They  defined  the  construct  as  “treats 
others  in  a  courteous,  diplomatic,  and  tactful  manner.  Provides  help  and  assistance  to  others. 
Works  effectively  as  a  team  member”  (p.  33). 

Agreeableness  is  one  of  the  “Big  Five”  personality  factors  assessed  in  the  NEO-PI  (Costa  & 
McRae,  1992).  According  to  Costa  and  McRae,  agreeableness  is  a  trait  in  which  it  is  desirable  to 
strike  a  balance  between  too  much  and  too  little.  Highly  agreeable  people  may  be  too  willing  to 
give  in  to  others  to  avoid  conflict,  while  those  who  are  low  in  agreeableness  may  come  across  as 
mde  or  abrupt,  stubborn,  or  tough-minded.  To  clarify  the  desirable  aspects  of  the  characteristic. 
Project  A  (Hough,  Barge,  &  Kamp,  2001)  defined  agreeableness  as  “The  degree  of  pleasantness 
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versus  unpleasantness  exhibited  in  interpersonal  relations.  Is  tactful,  helpful,  and  not  defensive, 
versus  touchy,  defensive,  alienated,  and  generally  contrary”  (p.  121);  Sager  and  Russell  (2003) 
adopted  that  definition  in  the  Select21  project. 

Knapp  et  al.  (2002)  identified  a  third  KSA  related  to  interpersonal  skills  -  concern  for  Soldiers’ 
quality  of  life.  The  population  of  interest  in  that  investigation  was  Army  non-commissioned 
officers  (NCOs),  so  this  KSA  was  quite  appropriate.  This  KSA  was  defined  as  “Is  aware  of 
subordinates’  off-duty  needs  and  constraints.  Is  sensitive  to  others’  priorities,  interests,  and 
values,  and  tries  to  assist  subordinates  in  making  their  personal  and  family  life  better”  (p.  6).  One 
of  the  goals  of  Phase  I  of  this  SBIR  was  to  identify  I/I  KS  As  for  Soldiers  in  a  range  of  pay 
grades,  which  will  include  NCOs  and  lower  pay  grades.  It  is  reasonable  to  expect  that  Soldiers 
with  strong  interpersonal  skills  would  also  be  concerned  for  the  quality  of  life  experienced  by 
their  colleagues,  even  though  that  concern  might  be  more  circumscribed  than  it  is  for  NCOs. 

Ford  et  al.  (2000)  concluded  that  military  operations  are  likely  to  become  more  frequent  in  the 
FF.  U.S.  forces  are  also  likely  to  respond  to  ongoing  terrorist  attacks,  as  has  happened  in 
Afghanistan  and  Iraq.  In  addition,  while  on  these  deployments,  Soldiers  are  likely  to  be  part  of 
coalition  forces  that  include  Soldiers  from  other  countries.  These  types  of  deployments  will 
require  that  Soldiers  remain  in  foreign  countries  or  in  contact  with  Soldiers  from  other  nations 
for  relatively  long  periods  of  time.  To  be  most  effective  in  their  role,  it  will  be  important  that 
they  understand  and  respect  the  customs  and  mores  of  the  country  where  they  are  stationed. 

Klein,  Pongonis,  and  Klein  (2000)  discuss  national  cultural  differences  and  the  KSAs  needed  by 
Soldiers  to  work  in  a  multi-cultural  environment.  They  conclude  that  the  ability  to  work  with 
team  members  from  different  cultures  involves  more  than  simply  accepting  that  there  are  cultural 
differences.  In  their  view,  multicultural  team  members  are  successful  to  the  extent  that  they 
attempt  to  understand  different  ways  of  thinking  and  approaching  problem  solving  -  to  see 
through  another  person’s  “cultural  lens”  (p.  1). 

In  addition  to  deployments  to  other  countries.  Soldiers  will  increasingly  find  the  demographic 
composition  of  the  Army  becoming  more  diversified.  In  the  20th  century,  a  new  Soldier  might 
find  him/herself  in  a  squad  with  Soldiers  from  across  the  U.S.,  from  varying  socioeconomic 
backgrounds  and  ethnicities.  Fueled  by  Soldiers  from  immigrant  families  and  Soldiers  who  are 
not  yet  American  citizens,  the  face  of  the  Army  will  continue  to  change.  Thus,  it  is  increasingly 
important  that  Soldiers  accept  and  are  tolerant  of,  if  not  embracing,  the  diversity  of  those  with 
whom  they  work  daily.  The  definition  of  cultural  tolerance  used  by  Ford  et  al.  (2000),  Knapp  et 
al.  (2002),  and  Sager  and  Russell  (2003)  emphasizes  the  need  to  be  tolerant  and  understanding  of 
those  from  other  cultures  and  social  backgrounds  during  interactions  with  Army  personnel  and 
foreign  nationals. 

It  is  inevitable  that  any  group  that  works  together  will  experience  some  kind  of  conflict.  Stevens 
and  Campion  (1994)  claimed  that  conflict  was  not  only  inevitable,  but  also  necessary  for  a  team 
to  “sense  the  need  for  change  or  draw  attention  to  problem  areas”  (p.  507).  The  hallmark  of 
effective  teamwork  is  being  able  to  resolve  conflicts  without  allowing  it  to  hurt  performance. 
Medsker  and  Campion  (1997)  recognized  the  importance  of  conflict  resolution  skills.  Their 
definition  included  (a)  the  ability  to  recognize  and  encourage  desirable  team  conflict  but 
discourage  undesirable  team  conflict,  (b)  the  ability  to  recognize  the  type  and  source  of  conflict 
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confronting  the  team  and  to  implement  an  appropriate  resolution  strategy,  and  (c)  the  ability  to 
employ  a  win-win  negotiation  strategy  rather  than  a  win-lose  strategy. 

Researchers  have  identified  two  general  types  of  conflict  management  processes  (Marks, 
Mathieu  &  Zaccaro,  2001;  Tjosvold,  1985).  A  preemptive  process  requires  team  members  to 
prevent  and  control  conflict  before  it  occurs.  For  example,  team  members  can  set  a  team  norm 
that  encourages  the  use  of  a  cooperative  approach  rather  than  competitive  approach  to  conflict 
resolution  (Tjosvold,  1985).  Team  members  can  also  clarify  some  rules  a  priori  about  how  they 
should  handle  conflict  situations.  In  a  reactive  process,  team  members  deal  in  a  constructive  way 
with  conflict  that  has  occurred.  Negotiation,  compromising,  and  willingness  to  accept  different 
opinions  are  some  often  used  strategies  for  reactive  conflict  management  (Marks  et  al.,  2001) 

Teamwork 

Although  the  Army  advertises  itself  as  “An  Army  of  One,”  that  slogan  refers  to  the  fact  that  the 
Army  considers  all  Soldiers  to  be  members  of  the  same  team.  That  feeling  is  inculcated  into 
Soldiers  during  Basic  Combat  Training  and  reinforced  at  all  levels.  Teamwork  has  long  been  a 
hallmark  of  the  Army.  The  FF  literature  (U.S.  Army,  2002)  indicates  that  this  team  orientation 
will  continue  to  be  a  strong  force  for  Soldiers  in  the  future.  Soldiers  will  need  to  be  able  to 
depend  on  their  buddies  during  stressful  campaigns  for  both  physical  and  emotional  support. 

Cannon-Bowers  et  al.  (1995)  identified  team  competencies  that  include  a  number  of  KSAs 
recognized  as  team  membership  skills,  such  as  interpersonal  cooperation,  morale  building, 
intermember  assistance,  and  intrateam  reinforcement.  They  also  identified  collective  orientation 
(belief  in  placing  team  goals  above  individual  goals)  and  belief  in  the  importance  of  teamwork  as 
critical  to  working  effectively  in  teams.  Along  the  same  lines,  Knapp  et  al.  (2002)  identified 
selfless  service  orientation  and  fostering  adaptive  teamwork  as  skills  that  support  effective 
teamwork;  and  Sager  and  Russell  (2003)  included  teamwork  orientation  among  the  KSAs  for 
entry-level  Soldiers. 

Rasker  (2002)  reviewed  the  teamwork  literature  through  2001  and  described  the  teamwork  KSAs 
that  have  been  enumerated  in  a  number  of  publications.  In  his  summary,  he  identified  four 
critical  behaviors  associated  with  team  performance:  monitoring  team  performance,  giving  intra¬ 
team  feedback,  communicating,  and  providing  backup.  He  identified  team  orientation,  team 
identity,  and  collective  behavior  as  important  KSAs  in  team  performance. 

Giving  and  receiving  feedback  is  one  major  mechanism  that  links  team  members’  task  activities 
together  (Cannon-Bowers  et  al.,  1995;  Marks,  Mathieu,  &  Zaccaro,  2001).  By  actively  seeking 
and  providing  feedback,  a  team  member  knows  what  activities  other  members  are  conducting, 
how  much  progress  they  have  made,  if  they  have  problems  or  need  help,  and  if  he/she  needs  to 
adjust  his/her  work  pace  to  coordinate  with  other  team  members. 

Among  Medsker  and  Campion’s  (1997)  teamwork  clusters  is  collaborative  problem  solving, 
which  includes  (a)  being  able  to  identify  when  group  problem  solving  is  appropriate  and  using 
the  technique  appropriately  and  (b)  recognizing  the  obstacles  to  collaborative  problem  solving 
and  removing  them. 
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Team  Leadership 

Any  team,  whether  by  design  or  accident,  might  find  itself  in  a  situation  that  requires  a  large 
degree  of  self-management  (Cannon-Bowers  et  al.,  1995;  Hackman,  1986).  Thus,  it  is  important 
for  both  team  members  and  team  leaders  to  exercise  leadership  such  as  planning  and  organising 
tasks,  assigning  tasks  among  team  members,  acting  as  a  role  model,  and  giving  directions  and 
suggestions  to  other  members  (Goodman,  Devadas,  &  Hughson,  1988;  Stevens  &  Campion, 
1994).  Team  leadership  includes  the  direction  and  structure  provided  by  formal  leaders  as  well  as 
by  other  team  members  (Larson  &  LaFasto,  1989).  Most  existing  team  or  teamwork  models 
(e.g.,  McIntyre  &  Dickinson,  1992;  Militello,  Kyne,  Klein,  Getchell,  &  Thordsen,  1999;  Nieva 
Fleishman,  &  Rieck,  1978;  Stevens  &  Campion,  1994)  include  team  management  or  team 
leadership  as  an  important  component  of  teamwork.  Rasker  (2002)  identified  several  dimensions 
that  are  related  to  performance,  including  the  ability  to  monitor  and  give  feedback,  and 
leadership  and  team  management  skills.  Taking  responsibility  for  giving  and  receiving  feedback 
is  related  to  the  team  leadership  KSA  identified  by  Knapp  et  al.  (2002)  as  being  important  for 
NCOs.  However,  it  is  not  necessary  to  be  a  supervisor  to  use  or  develop  leadership  skills.  The 
Army  is  also  eager  to  identify  and  reinforce  leadership  potential  (U.S.  Army,  2002),  which 
makes  characteristics  such  as  acting  as  a  role  model  and  willingness  to  help  others  important 
characteristics  of  all  FF  Soldiers. 

Teams,  particularly  in  the  Army,  must  often  work  in  rapidly  changing  and  uncertain 
environments.  This  requires  that  team  members  be  able  to  adjust  themselves  to  different  roles, 
learn  new  ways  to  perform  their  jobs,  and  learn  quickly  how  to  cooperate  with  others  (Black, 
1990;  Noe  &  Ford,  1992).  Therefore,  adaptability  is  often  considered  as  one  important 
component  of  teamwork  (Cannon-Bowers  et  al.,  1995;  Eby,  Meade,  Parisi,  &  Douthitt,  1999; 
Morgan,  Glickman,  Woodard,  Blaiwes,  &  Salas,  1986;  Sundstrom,  de  Meuse,  &  Futrell,  1990). 
Pulakos,  Arad,  Donovan,  and  Plamondon  (2000)  developed  a  conceptual  model  that  described 
four  generic  dimensions  of  adaptability:  (a)  solving  problems  creatively,  (b)  learning  new  work 
tasks,  technologies  and  procedures  actively,  (c)  adjusting  interpersonal  styles  and  behaviors  to 
work  with  new  coworkers  flexibly  to  achieve  goals,  and  (d)  performing  effectively  in  different 
cultural  and  physical  environments. 

Social  Perceptiveness 

Kihlstrom  and  Cantor  (2000)  define  social  intelligence  as  the  ability  to  understand  people  and  to 
act  effectively  in  human  relations.  They  propose  five  dimensions  to  describe  Social  Intelligence: 
(a)  interest  and  concern  for  other  people,  (b)  social  performance  ability,  (c)  empathic  ability,  (d) 
emotional  expressiveness  and  sensitivity  to  others’ emotional  expressions,  and  (e)  social  anxiety 
and  lack  of  social  self-efficacy.  Social  intelligence  comprises  the  ability  to  be  aware  of  one’s  own 
and  others’  emotions  and  to  use  that  awareness  to  guide  thinking  and  actions.  This  KSA  is  also 
referred  to  as  social  perceptiveness  (Knapp  et  al.,  2002;  Sager  &  Russell,  2003).  Interpersonal 
intelligence  is  defined  as  the  ability  to  understand  other  people:  what  motivates  them,  how  they 
work,  and  how  to  work  cooperatively  with  them  (Young,  1996). 
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Communication 


Skills  related  to  this  dimension  identified  in  the  NC021  research  effort  by  Ford  et  al.  (2000)  are 
principally  focused  on  effectively  communicating  to  subordinates:  (a)  directing,  monitoring,  and 
supervising  subordinates,  (b)  motivating,  leading,  and  supporting  individual  subordinates,  and  (c) 
training  others.  Knapp  et  al.  (2002)  identified  the  following  three  KSA  areas  that  include 
communication:  (a)  oral  communication,  (b)  writing  skill,  and  (c)  training  others.  Medsker  and 
Campion  (1997)  found  communication  skills  to  be  predictive  of  teamwork  performance  ratings. 
In  their  view,  communication  skills  include  the  ability  to  (a)  communicate  openly  and 
supportively,  (b)  listen  non-evaluatively  and  use  active  listening  techniques  when  appropriate, 

(c)  coordinate  verbal  and  nonverbal  messages  to  use  them  most  effectively,  and  (d)  engage  in 
small  talk  and  ritual  greetings,  recognizing  their  importance.  Rasker’s  (2002)  review  of  the  team 
performance  literature  also  identified  communicating  as  one  of  the  critical  behaviors  associated 
with  team  performance  ratings. 

Although  letter  writing  is  on  the  wane,  written  communication  is  not  losing  its  importance. 

People  increasingly  communicate  electronically  via  email  or  instant  messages.  Although  these 
media  offer  ample  opportunity  to  communicate,  they  also  present  the  problem  of  conveying 
information  with  little  context  to  help  interpret  the  meaning  of  a  message.  Thus,  it  is  more  of  a 
challenge  to  convey  the  spirit  of  the  communication  in  fewer  words. 

The  list  of  nineteen  I/I  KSAs  developed  as  a  result  of  the  literature  review  is  shown  in  Table  1. 

Determination  of  Interaction/Interpersonal  KSAs  for  the  FF  Soldier 

The  literature  provided  us  with  many  variations  on  some  common  themes,  so  the  task  fell  to  us  to 
sort  and  combine  them  into  meaningful  KSAs.  In  some  cases,  traits  or  capabilities  that  were 
labeled  differently  appeared  to  describe  the  same  construct  (e.g.,  multi-cultural  teamwork  & 
cultural  tolerance,  intermember  assistance  and  team  orientation),  so  these  traits  were  aggregated 
into  a  single  dimension.  In  other  instances,  KSAs  seemed  logically  related  to  each  other  but 
distinct  enough  to  stand  on  their  own  or,  as  mentioned  previously,  it  made  sense  to  break  a 
complex  KSA  such  as  oral  communication  into  some  of  its  component  parts  (e.g.,  active 
listening  and  nonverbal  skills)  to  emphasize  their  importance  as  I/I  skills. 

However,  as  the  team  began  to  work  with  the  KSAs  and  think  about  how  they  could  be  assessed, 
we  identified  two  potential  problems  with  trying  to  measure  each  of  them  individually.  First, 
there  is  a  significant  degree  of  overlap  between  some  of  the  KSAs  (e.g.,  “Relate  to  and  support 
peers”  with  “Amicability”).  Second,  not  all  of  the  KSAs  have  sufficient  research  behind  them  to 
support  development  of  measures  (e.g.,  active  listening).  So,  we  re-categorized  the  KSAs  into 
what  we  refer  to  as  the  Core  I/I  KSAs.  Some  of  the  KSAs  stand  on  their  own  while  others  are 
subsumed  under  a  broader  rubric.  The  KSAs  have  been  reorganized  as  shown  in  Table  2. 
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Table  L  Interpersonal/Interaction  KSAs  Identified  in  the  Literature^  Review _ 

Ability  to  Relate  to  and  Support  Peers.  The  degree  to  which  the  individual  treats  peers  in  a  courteous,  respectful, 
and  tactful  manner.  Provides  help  and  assistance  to  others.  Backs  up  and  fills  in  for  others  when  needed.  Works 
effectively  as  a  team  member. 

Amicability.  The  degree  of  pleasantness  versus  unpleasantness  exhibited  in  interpersonal  relations.  Exhibits 
goodwill  towards  others  and  an  absence  of  antagonism.  Is  tactful  and  helpful  rather  than  defensive,  touchy,  and 
generally  contrary. 

Concern  for  Soldier  Quality  of  Life.  Is  sensitive  to  others’  priorities,  interests,  and  values,  and  tries  to  assist  them 
in  making  their  personal  and  family  life  better. 

Conflict  Management.  The  degree  to  which  the  individual  encourages  and  supports  different  perspectives,  avoids 
harmful  conflict,  constructively  addresses  disagreements  that  undermine  team  performance,  and  does  not  allow 
conflicts  with  others  in  ways  that  preserve  good  relations  and  enhance  trust. 

Cultural  Tolerance.  The  degree  to  which  an  individual  demonstrates  toleraiice  and  understanding  of  individuals 
from  other  cultural  and  social  backgrounds,  both  in  the  context  of  the  diversity  of  U.S.  Army  personnel  and 
interactions  with  foreign  nationals  during  deployments  or  when  training  for  deployment. 

Dependability.  The  person’s  characteristic  degree  of  conscientiousness.  Is  disciplined,  well  organized,  planful,  and 
respectful  of  laws  and  regulations. 

Team  Orientation.  The  degree  to  which  an  individual  identifies  with  the  team  and  other  team  members  and  works 
to  boost  team  morale  and  increase  the  team  bond  by  creating  and  maintaining  a  supportive  work  environment; 
willingness  to  put  the  needs  of  the  team  ahead  of  personal  needs. 

Affiliation.  The  degree  of  sociability  that  one  exhibits.  Is  outgoing,  participative,  and  friendly  versus  shy  and 
reserved. 

Coordination.  The  ability  to  work  interdependently  to  reach  task  completion,  share  information  and  effort,  and 
work  together  with  others.  Can  adjust  own  time  and  work  activities  to  ensure  interdependent  tasks  are  completed 
effectively. 

Cooperativeness  in  Problem-Solving.  The  ability  to  take  advantage  of  multiple  perspectives  to  find  effective 
solutions  to  problems. 

Adaptability/Flexibility.  The  degree  to  which  an  individual  is  able  to  respond  to  rapidly  changing  situations  (e.g., 
assignments,  relocation,  new  Soldiers)  and  accept  new  roles. 

Social  Perceptiveness.  The  degree  to  which  an  individual  is  able  to  monitor  own  and  other’s  emotions,  discriminate 
among  them,  and  use  the  information  to  guide  one’s  thinking  and  actions,  allowing  one  to  work  cooperatively  with 
others.  Is  aware  of  how  own  behavior  impacts  others. 

Oral  Communication  Skills.  The  ability  to  speak  clearly  and  precisely  so  that  others  can  easily  understand.  The 
ability  to  adapt  speaking  style  and  comments  to  the  audience,  as  appropriate. 

Active  Listening.  The  ability  to  listen  non-evaluatively  to  instructions  and  related  messages,  focusing  on  the  person 
communicating.  Asks  and  responds  to  questions  as  appropriate.  Pays  attention  to  nonverbal  cues  to  help 
clarify/interpret  messages. 

Nonverbal  Skills.  The  ability  to  match  verbal  and  nonverbal  messages  when  speaking  to  clarify  and  enhance  the 
message.  The  degree  to  which  an  individual  can  accurately  interpret  the  nonverbal  signals  of  others. 

Written  Communication.  The  ability  to  write  clearly  so  that  message  is  understood  by  the  reader.  Is  sensitive  to 
the  limitations  of  written  communication  (e.g.,  email)  and  carefully  phrases  message  so  that  the  intent  can  be  clearly 
understood  by  the  receiver. 

Acts  as  a  Role  Model.  Exhibits  self-confidence  and  a  positive  attitude.  Presents  a  positive  and  professional  image 
of  self  and  the  Army  even  when  off  duty. 

Helping  Others.  The  ability  to  help  other  team  members  to  improve  performance.  Willingness  to  provide  assistance 
as  needed  and  to  guide  and  tutor  others  on  technical  matters. 

Task  Leadership.  Ability  to  help  keep  the  team  focused  on  the  team’s  assignment  or  mission,  working  with  team 
members  to  react  to  changes  and  to  ensure  that  conflicts  do  not  hinder  mission  achievement. 
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Table  2.  Core  Interpersonal/Interaction  KSAs 


I.  Relating  to  and  supporting  others 

A.  Ability  to  relate  to  and  support  peers 

B.  Amicability 

C.  Concern  for  Soldiers’  quality  of  life 

IE.  Conflict  management 

ID.  Cultural  tolerance 

IV.  Dependability 

V.  Teamwork 

A.  Team  orientation 

B.  Affiliation 

C.  Coordination 

D.  Cooperativeness  in  problem-solving 

VI.  Adaptability/Flexibility 

VH.  Social  Perceptiveness 

Vm.  Communication  ability 

A.  Oral  communication 

B.  Active  listening 

C.  Nonverbal  communication  skills 

D.  Written  communication 

IX.  Peer  Leadership 

A.  Acts  as  a  role  model 

B.  Helping  others 

C.  Task  leadership 


Research  And  Critique  Interpersonal/Interaction  KSA  Measures/Techniques 
Identify  Assessment  Measures  or  Techniques 

The  second  task  of  the  Phase  I  effort  was  to  identify  assessment  techniques  by  which  to  evaluate 
individuals  on  the  core  I/I  KSAs.  We  developed  a  list  of  assessment  methods  (see  Table  3)  and 
identified  measures  that  could  be  included  in  those  categories.  These  included  COTS  instruments 
(e.g.,  NEO-PI,  16PF)  and  measures  designed  for  previous  ARI  projects  (i.e.,  NC021  and 
Select21),  as  well  as  measures  such  as  computerized  simulations  and  role  plays  that  could  be 
developed  in  Phase  n. 

We  collected  information  related  to  the  measures  and  measurement  methods  through  related 
documentation  (e.g.,  technical  manuals,  research  reports).  In  addition,  information  in  the 
research  literature  and  the  experience  of  the  project  staff  were  used  to  identify  and  evaluate 
potential  measurement  methods. 

One  factor  to  bear  in  mind  is  that  research  has  shown  that  some  of  the  KSAs  we  plan  to  measure 
are  susceptible  to  response  distortion.  That  is,  individuals  who  are  motivated  to  present  a  false 
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impression  of  themselves  are  able  to  do  so.  For  that  reason,  we  favored  strategies  that  have  been 
shown  to  be  helpful  in  reducing  response  distortion.  It  will  also  be  necessary  to  evaluate  the 
potential  for  response  distortion  on  the  measures  that  are  subsequently  developed  or  used  in 
future  phases  of  this  work. 

Table  3.  Possible^  Assessment^  Methods _ 

Text-based 

Self-report  (fixed  response) 

Self-report  (free  response) 

Forced-choice 

Scenario-based  (fixed  response) 

Oral  interviews 

Situational,  behavior  description,  combination  or  other  structured 

Behavior  descriptions 

Combination  or  other  structured  format 

Clinical 

Simulations  (computer  based) 

High  fidelity  stimulus  and  response 
High  fidelity  stimulus  and  low  fidelity  response 
Low  fidelity  stimulus  and  high  fidelity  response 
Low  fidelity  stimulus  and  response 

Live  action 
Individual 
Group  simulations 
Role  play 

Real  life  behavior 

Performance  ratings 

Work  product  review _ 


Develop  a  KSA-by-Method  Measurement  Plan 

To  facilitate  selection  of  the  best  methods  for  assessing  the  I/I  KSAs  identified  in  the  Phase  I 
effort,  we  developed  a  KSA-by-Method  matrix.  In  the  original  matrix,  the  KSAs  were  listed  in 
the  left  column  and  all  the  potential  measure  types  were  listed  across  the  top.  While  this  was  an 
efficient  way  to  set  up  the  matrix  for  populating  it,  it  ended  up  with  more  empty  cells  than 
populated  cells.  Therefore,  we  captured  that  information  and  put  it  in  a  more  readable  format, 
presented  in  Appendix  A. 

When  that  task  was  completed,  the  next  step  was  to  assess  the  feasibility  of  using  each  type  of 
test  or  technique  to  assess  each  of  the  I/I  KSAs.  Each  measurement  method  was  rated  on  three 
criteria  to  determine  which  method  would  be  most  appropriate  for  measuring  which  KSA.  The 
criteria  used  in  evaluating  the  measurement  methods  were  as  follows: 

•  Appropriateness  of  Method  (AoM):  Whether  the  method  is  appropriate  for  use  for  the 
given  KSA,  to  what  degree  that  method  can  be  used  to  tap  the  KSA.  It  is  scored  as:  0  - 
Not  Appropriate;  1  -  Possibly  Appropriate,  2  -  Appropriate 
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•  Susceptibility  to  Faking  (F):  The  degree  to  which  the  method  can  be  easily  faked.  It  is 
scored  on  a  scale  of  1  to  5  with  1  indicating  highly  susceptible  to  faking  and  5  being  not 
susceptible  to  faking. 

•  Ease  of  Implementation  (Eol):  The  degree  of  difficulty  associated  with  actually  using  the 
method  to  gauge  the  KSA.  It  is  scored  on  scale  of  1  to  5  with  1  indicating  highly  difficult 
to  implement  (high  cost,  labor  intensive,  etc)  and  5  indicating  low  difficulty  in 
implementation. 


We  populated  the  KSA-by-Method  matrix  with  those  ratings.  Again,  the  matrix  is  too  unwieldy 
to  show  in  its  entirety,  but  Figure  1  shows  a  partial  depiction.  To  help  identify  the  final 
assessment  methods,  we  also  created  a  single  “Utility  to  the  Army”  (UA)  index  that  collapses 
information  from  the  ratings  listed  above.  It  provides  a  score  that  can  be  used  to  rank  order 
available  measurement  techniques. 

UA  scores  can  range  from  0  to  20,  with  higher  UA  scores  indicating  more  utility  to  the  Army. 
The  ideal  score  (20)  would  be  reached  by  a  measurement  technique  that  is  very  appropriate  for 
the  KSA,  very  easy  to  implement  (develop,  administer,  and  interpret),  and  highly  resistant  to 
faking.  It  seems  unlikely  that  any  measurement  technique  would  reach  that  score  as  more 
appropriate  measures  may  generally  be  more  “fakeable,”  and  many  appropriate  measures,  such 
as  group  role  plays,  appear  generally  to  be  more  difficult  to  implement  and/or  score.  For  more 
detailed  information  on  UA  calculations  see  Appendix  B. 

The  most  important  criterion  for  a  test  is  how  well  it  measures  the  KSA  of  interest,  which  we  are 
calling  appropriateness.  A  test  that  has  an  appropriateness  score  of  1  may  have  a  UA  that  is  equal 
to  or  higher  than  a  test  with  an  appropriateness  score  of  2,  if  it  is  more  resistant  to  faking  and 
easier  to  develop  and  administer.  This  tradeoff  is  important  to  consider  because  the  most  valid 
tests  are  frequently  either  more  fakeable  or  more  difficult  to  develop  and  implement.  Table  4 
presents  the  top  three  highest  UA  measurement  methods  for  each  of  the  I/I  KSAs.  The  KSAs 
presented  in  bold  type  are  the  core  KSAs.  Appendix  B  contains  the  written  summary  of  the 
exercise,  describing  the  rationale  behind  each  rating. 

The  KSA-by-method  matrix  and  associated  ratings  were  used  as  supporting  aids  in  a  project 
team  meeting  to  develop  an  I/I  KSA  measurement  plan.  In  this  brainstorming  meeting, 
participants  nominated  various  measurement  methods  for  inclusion  in  an  experimental 
assessment  battery. 

There  are  many  existing  COTS  products  that  are  often  used  to  assess  VI  KSAs.  Meeting 
participants  agreed  to  exclude  most  COTS  measures  because  of  cost  considerations  and  difficulty 
and  time  delay  associated  with  scoring  these  assessments.  We  believe  that  we  can  develop  new 
measures  or  adapt  existing  measures  for  this  project  that  will  serve  our  needs  without  the 
constraints  involved  with  COTS  assessments. 

Although  there  are  a  variety  of  contexts  in  which  these  measures  might  be  used,  we  assumed  that 
one-on-one  assessment  procedures  would  not  be  practical  and  that  field  personnel  should  not  be 
expected  to  learn  and  use  complicated  scoring  procedures.  We  favored  measurement  methods 
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that  would  complement  each  other  and,  in  combination,  would  represent  technical  and 
technological  innovations  in  assessment. 


Table  4.  Summary  ^Measurement  Methods  with  Greatest  Utility  to  the  Army 


KSA 

Highest  UA  Method 

2nd  Highest  UA 
Method 

3rd  Highest  UA  Method 

Relating  to  and 

Supporting  Others 

Relate  to  and  Support  Peers 

Real  life  behavior  (9-18) 

Live  Action  (10-12) 

Simulation  (10-12) 

Amicability 

Live  Action  (10) 

Simulation  (8-10) 

Concern  for  Soldier 

Live  Action  (10-12) 

Self-report  (6-8) 

Real  life  behavior  (7) 

Quality  of  Life 

Conflict  Management 

Simulation  (10-12) 

Live  Action  (8-12) 

Real  life  behavior  (7-8) 

Cultural  Tolerance 

Interviews  (8-12) 

Live  action  (8) 

Real  life  behavior  (7-8) 

Dependability 

Real  life  behavior  (16) 

Interviews  (5) 

Live  action  (4-5) 

Team  Orientation 

Team  Orientation 

Interviews  (10-12) 

Live  action  (10) 

Self-report  (7-10) 

Affiliation 

Live  Action  (10-12) 

Simulation  (10-12) 

Real  life  behavior  (8) 

Coordination 

Live  action  (10-14) 

Simulation  (10-12) 

Real  life  behavior  (7-8) 

Social  Perceptiveness 

Simulation  (10-12) 

Live  action  (8-12) 

Interviews  (6-7) 

Adaptabiiity/Flexibility 

Self-report  (5-14) 

Live  Action  (10-12) 

Interviews  (10) 

Communicate  Orally 

Active  Listening 

Live  action  (12-16) 

Interviews  (12-14) 

Simulation  (8) 

Non-verbal  skills 

Interviews  (16) 

Live  action  (12-14) 

Oral  Communication  skills 

Interviews  (16) 

Live  action  (16) 

Real  life  behavior  (0-8) 

Written  communication 
skills 

Leading  Others 

Acts  as  a  role  model 

Real  life  behavior  (8-10) 

Real  life  behavior  (6-16) 

Live  Action  (12) 

Interviews  (6) 

Helping  others 

Simulation  (8-10) 

Live  action  (6-10) 

Self-report  (4-7) 

Task  leadership 

Live  action  (14-16) 

Real  life  behavior  (7) 

Simulation  &  Self-report 

Cooperativeness  in 

Live  action  (8-10) 

Real  life  behavior 

(5-6) 

Interviews  (4) 

problem  solving 

(7-8) 

The  brainstorming  meeting  resulted  in  a  roughed-out  concept  of  a  2-stage  assessment  process 
with  multiple  measures  at  each  stage.  Following  the  meeting,  a  more  detailed  measurement  plan 
was  devised. 
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Method 

[  SELF  REPORT 

Interviews 

SunttUtionr 

KSA 

. I .  t . 

Fixed  Response 

Free  Response 

Forced  Choice 

Scenario  Rased 
Fixed  Response 

Structured 

Interview 

Omical 

High  Fidelity 
Stimulus;  & 
Response 

High  Fidel 
Stimulus  - 1 
Fid.  Respo 

Figure  1.  Example  of  KSA-by-method  matrix  populated  with  ratings  on  the  utility  criteria. 


Develop  Innovative  Assessment  Concepts 

The  next  task  was  to  complete  a  draft  design  of  the  I/I  KSA  assessment  battery.  This  battery 
includes  a  combination  of  simulations  and  self-report  measures,  a  work  sample,  and  a  live  action 
assessment.  We  think  the  measures  in  this  set  complement  each  other  well,  assess  the  constructs 
of  interest,  and  will  be  reasonably  easy  to  implement  with  the  technology  available  to  us.  The 
proposed  2-stage  assessment  process  is  illustrated  in  Figure  2.  Stage  1  will  focus  on  whether  an 
examinee  has  the  knowledge  of  what  should  be  done  when  interacting  with  others;  Stage  2  will 
assess  whether  the  examinee  can  demonstrate  skill.  For  example,  a  person  may  know  that  it  is 
inappropriate  to  interrupt  a  speaker  before  she  is  finished,  but  may  still  do  so  when  interacting 
with  others. 

In  this  section,  we  describe  in  more  detail  the  types  of  assessments  we  plan  to  use.  Particularly 
given  the  experimental  nature  of  some  of  these  ideas,  a  detailed  measurement  plan  will  evolve  as 
we  begin  prototyping  the  measures. 
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Stage  1  Measures 


Biodata 

Instrument 

(self-report) 


Situational  Judgment  Test/ 
Action  Exam 
(high-fidelity  simulation) 


Stage  2  Measures 


Leaderless  Group 

Semi-Structured 

Writing 

Discussion 

Interview 

Skill  Test 

(live  action  group  simulation) 

(oral  interview) 

(high-fidelity  simulation) 

Figure  2.  The  proposed  Phase  11  assessment  battery. 


The  Proposed  Assessment  Battery 


Stage  1  Measures 

In  Stage  1,  we  propose  to  adapt  an  existing  biodata  instrument  for  our  use,  and  develop  either  a 
situational  judgment  test  (SJT)  or  an  action  exam.  The  SJT  and  action  exam  can  both  be  designed 
to  assess  the  same  KS  As.  The  SJT  is  a  commonly  used  instrument  that  has  a  good  body  of  research 
behind  it,  and  we  have  solid  experience  in  developing  this  type  of  test.  While  it  has  traditionally 
been  a  paper-and-pencil  test,  it  can  easily  be  adapted  to  a  computer  simulation.  The  action  exam, 
on  the  other  hand,  is  a  fairly  innovative  assessment.  It  is  a  student  evaluation  technique  that  was 
developed  by  instructors  to  determine  whether  students  were  able  to  apply  the  “book  learning” 
from  class  to  a  role  play  or  videotaped  scenario  (Keleman,  Garcia,  &  Lovelace,  1990). 

Biodata  Measure 

A  biodata  inventory  is  a  self-report  instrument  that  contains  questions  that  are  relatively 
objective  and  readily  verifiable,  such  as  the  amount  and  nature  of  education,  job  experiences, 
special  skills,  hobbies,  and  recreational  activities.  It  is  not  unusual  for  a  biodata  inventory  to  be 
used  to  obtain  some  of  the  same  types  of  information  under  uniform  conditions  and  in  situations 
where  it  would  not  be  feasible  to  conduct  individual  interviews.  The  rationale  behind  the  use  of 
biodata  is  that  the  activities,  interests,  and  behaviors  one  has  exhibited  in  the  past  are  predictive 
of  what  one  will  do  in  the  future  (Owens,  1976). 

Biodata  inventories  are  appealing  in  testing  situations  because  previous  research  has  indicated 
that  they  tend  to  have  very  low  adverse  impact  when  items  are  screened  in  advance  (Mumford  & 
Stokes,  1992).  Furthermore,  because  biodata  forms  measure  characteristics  not  captured  by 
standard  ability  tests,  they  tend  to  provide  incremental  validity  over  that  obtained  with  cognitive 
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measures  (Hunter  &  Hunter,  1984;  Mitchell,  1994).  Biodata  measures  have  been  found  to  be 
consistently  good  predictors  of  a  variety  of  performance  measures  for  many  different  jobs  (e.g., 
Kilcullen,  White,  Zacarro,  &  Parker,  2000;  Owens,  1976).  The  validity  of  biographical 
information  or  data  (or  biodata)  has  been  demonstrated  in  several  previous  studies  (Hunter  & 
Hunter,  1984;  White  &  Young,  1998;  Young,  Heggestad,  Rumsey,  &  White,  2000).  Previous 
research  has  also  shown  that  biodata  scales  can  be  used  to  measure  personality  constructs,  have 
higher  criterion-related  validity,  and  are  less  easily  faked  than  traditional  self-report  personality 
assessments  (Kilcullen,  White,  Mumford,  &  Mack,  1995).  In  comparison  to  alternative 
measures,  biodata  forms  are  feasible  for  group  administration  and  scoring  at  a  lower  cost  and  are 
generally  accepted  by  employers  and  the  courts. 

Recent  emphases  in  biodata  research  have  been  on  the  understanding  of  the  constructs 
underlying  biodata  measures  (Mitchell,  1994;  Stokes,  2001).  Instead  of  empirically  determining 
which  sets  of  biodata  items  might  distinguish  between  job  incumbents  who  perform  well  and 
those  who  do  not  perform  well,  the  goal  is  to  write  items  targeted  to  specific  KSAs  required  for 
successful  job  performance.  Biodata  items  have  two  essential  characteristics:  (a)  people  are 
asked  to  recall  and  report  behavior  and  experiences,  and  (b)  items  refer  to  behavior  and 
experiences  occurring  in  specific  situations  to  which  individuals  are  likely  to  have  been  exposed. 
For  example,  if  the  KS  A  required  is  achievement  motivation,  a  biodata  item  might  consist  of  the 
question,  “How  important  was  it  for  you  to  get  A’s  in  school?”  A  question  for  the  same  domain 
but  tailored  to  an  adult  might  be  “How  important  has  it  been  to  you  to  receive  excellent 
performance  evaluations  on  your  job?” 

Biodata  items  reflect  various  situational  and  behavioral  scenarios  including  situational  exposure, 
situational  choice,  behavior  in  a  situation,  reactions  to  situations,  others’  reactions  to  a  situation, 
and  outcomes  associated  with  personal  exposure.  Items  can  be  either  direct  (e.g.,  What  was  the 
average  grade  you  received  on  writing  assignments  during  college?)  or  indirect  (e.g..  How  many 
books  were  around  your  home  when  you  were  growing  up?).  Biodata  items  must  take  into 
account  the  historic  condition,  social  expectation,  and  age  of  the  population.  Multiple  items  are 
written  for  each  KSA  or  construct  of  interest;  it  is  through  the  use  of  multiple  items  that  a 
meaningful  pattern  emerges. 

While  we  think  that  a  biodata  instrument  will  provide  valuable  information,  we  do  not  plan  to 
develop  the  instrument  from  scratch.  A  traditional,  empirically-based  instrument  is  too  expensive 
to  develop  and  likely  to  lose  predictive  validity  over  time.  In  addition,  rationally-based  biodata 
instruments  tend  to  work  better  in  our  experience.  Therefore,  we  propose  to  use  the  instrument 
currently  under  review  and  development  in  Select21.  This  instrument,  tentatively  named  the 
TAP/BIQ  Amalgam  (TBA)  is  composed  of  selected  scales  from  the  Test  of  Adaptable  Personality 
(TAP)  and  the  NC021  Biographical  Information  Questionnaire  (BIQ)  (Knapp  et  al.,  2002). 

The  TAP  is  a  20-minute  biodata  assessment  that  has  demonstrated  criterion-related  validity  in 
operational  use  for  Special  Forces  Soldiers.  As  work  on  the  instrument  for  Select21  progressed, 
the  project  staff  decided  to  use  only  some  of  the  TAP  scales  and  supplement  them  with  scales 
from  the  BIQ,  which  is  a  conglomerate  biodata  instrument  administered  during  the  NC021 
project  (cf.  Putka,  Kilcullen,  &  White,  2003)  and  comprising  ARI’s  Assessment  of  Right 
Conduct.  KSAs  that  could  be  measured  with  the  TBA  include: 
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•  Adaptability/Flexibility 

•  Cultural  Tolerance 

•  Dependability 

•  Social  Perceptiveness 

•  Team  Orientation 

Situational  Judgment  Test  (SJT) 

SJTs,  which  have  been  in  use  for  decades,  contain  a  series  of  job  situations  or  problems  typically 
presented  in  writing  (Motowidlo,  Dunnette,  &  Carter,  1990).  There  are  two  parts  to  an  SJT  item. 
The  scenario,  or  stem  of  the  question,  describes  the  situation  and  provides  a  problem  that 
requires  a  decision  to  be  made.  The  response  options  provide  realistic  actions  for  handling  the 
problem  situation.  Unlike  typical  multiple-choice  tests,  there  is  not  one  correct  answer  and 
several  wrong  answers.  Rather,  each  option  is  a  plausible  way  of  handling  the  situation,  with 
some  options  being  better  than  others. 

Well-written  situations  are  actual  events  that  one  could  encounter  on  a  job.  They  are  relatively 
brief  (two  to  four  sentences)  that  require  a  decision  on  the  part  of  the  respondent,  although  the 
best  answer  should  not  be  obvious.  The  situation  should  not  be  one  that  requires  training  to 
recognize  the  best  answer.  Each  situation  should  relate  to  one  of  the  competencies  that  the  SJT  is 
intended  to  measure. 

SJTs  have  been  used  to  assess  an  array  of  constructs,  including  conflict  resolution  (Drasgow, 
Olson-Buchanan,  &  Moberg,  1999),  managerial  skills  (Motowidlo,  Hanson,  &  Crafts,  1997; 
Waugh,  2002;  Waugh  &  Sticha,  2001),  and  technical  skills  (Hanson,  Borman,  Mogilka, 

Manning,  &  Hedge,  1999).  An  example  of  a  situational  item  adapted  from  a  paper  by  Waugh, 
Putka,  and  Sager  (2002)  is  presented  in  Figure  3. 


One  of  your  fellow  employees  feels  like  he  doesn’t  have  to  pitch  in  and  do  the  work  that  you 
were  all  told  to  do.  What  should  you  do? 

1 .  Explain  to  the  individual  that  he  is  part  of  a  team  and  needs  to  pull  his  weight. 

2.  Report  him  to  your  supervisor. 

3.  Find  out  why  he  feels  he  doesn’t  need  to  pitch  in. 

4.  Keep  out  of  it;  this  is  something  for  your  boss  to  notice  and  correct. 

*  adapted  from  Waugh  et  al.,  2002  ”* '  ’ 

Figure  3.  Example  situational  judgment  item. 

SJTs  have  become  very  popular  assessment  instmments  for  several  reasons.  First,  they  assess 
knowledge  and  skills  that  cannot  be  addressed  with  traditional  multiple-choice  test  formats  and 
yield  reasonably  high  estimated  validities  for  predicting  job  performance.  McDaniel,  Bruhn- 
Finnegan,  Morgeson,  Campion,  and  Braverman  (1997)  conducted  a  meta-analysis  that  found  the 
validity  of  SJTs  (corrected  for  restriction  of  range  and  criterion  unreliability)  was  .56.  Second, 
this  type  of  test  is  relatively  easy  and  inexpensive  to  score.  A  third  advantage  is  that  differences 
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in  mean  scores  between  racial  subgroups  are  typically  smaller  than  that  found  with  cognitive 
ability  tests.  Motowidlo  et  al.  (1990)  reported  standardized  mean  differences  between  African- 
Americans  and  Caucasians  of  .14  and  .29.  Schmitt  et  al.  (1994)  found  moderate  mean  differences 
(.41)  between  African-Americans  and  Caucasians,  no  differences  between  Caucasians  and 
Hispanics,  and  a  small  mean  difference  between  males  and  females.  Fourth,  examinees  perceive 
SJTs  to  be  face  valid  (Motowidlo  et  al.,  1990)  and  appropriate  as  a  job-relevant  assessment.  Face 
validity  has  been  shown  to  produce  positive  examinee  reactions  (Bauer,  Maertz,  Dolin,  & 
Campion,  1998;  Ployhart  &  Ryan,  1998).  The  perception  of  face  validity  and  positive  examinee 
reactions  appear  to  play  a  role  in  test  taking  motivation  and  the  magnitude  of  subgroup 
differences  (Chan,  Schmitt,  DeShon,  Clause,  &  Delbridge,  1997). 

We  believe  that  the  situational  judgment  test  is  a  good  alternative,  because  once  scoring  keys  are 
developed,  SJTs  are  easily  administered  and  machine  scored.  They  are  also  easily  adapted  to 
automation.  This  type  of  instrument  can  measure  important  KSAs  that  will  help  to  ensure  that 
examinees  are  properly  screened  for  the  Stage  2  assessments. 

Action  Exam 

In  their  early  work  with  action  exams,  Keleman  and  his  colleagues  (Bigelow,  1991;  Keleman,  et 
al.,  1990;  Keleman,  Lovelace,  &  Garcia,  1991)  used  role  plays  to  evaluate  their  students’  ability 
to  convert  cognitive  learning  to  an  interpersonal  interaction.  They  were  concerned  when  students 
who  had  done  well  in  the  classroom  had  difficulty  acting  in  new  interaction  situations.  They 
concluded  that  classroom  presentations  and  textbooks  present  problems  in  neat  packages  for 
which  there  is  clearly  an  answer.  Real  world  interactions  are  often  ill-defined  and  complex, 
however,  leaving  the  problem  solver  to  define  the  problem  and  come  up  with  a  solution.  Because 
interpersonal  interactions  are  a  large  part  of  problem  solving,  these  researchers  developed  a 
system  that  would  allow  them  to  assess  how  well  their  students  could  perform  in  such  situations 
and  also  provide  focused  feedback  to  the  student. 

Typically,  they  presented  the  student  with  a  situation  (e.g.,  the  student  is  a  supervisor  in  a  service 
organization  and  has  just  seen  a  subordinate  respond  to  a  group  of  customers  in  a  way  that  made 
them  angry;  the  instructor  is  the  subordinate  who  has  an  explanation)  and  then  set  the  stage  for  a 
discussion  between  the  two  parties.  The  instructor  then  rated  the  effectiveness  with  which  the 
student  demonstrated  skill  in  managing  “Interaction  Flow,”  “Interaction  Outcomes,”  and 
“Communication.” 

Interaction  Flow  includes  such  things  as  opening  the  discussion,  discussing  needs  of  both  parties, 
and  mutually  developing  a  solution.  Interaction  Outcomes  focus  on  whether  the  solution  took 
common  interests  into  account,  whether  the  outcome  is  tied  to  objective  criteria,  and  whether  it 
will  maintain  or  improve  the  relationship.  The  rating  for  Communication  includes  effective  use 
of  nonverbal  behavior,  being  problem  focused,  active  listening,  style  flexibility,  and  maintaining 
composure  (Keleman  et  al.,  1990). 

If  we  decide  to  include  an  action  exam  in  the  test  battery,  we  would  quite  likely  take  a  different 
approach.  The  Stage  1  assessments  will  need  to  be  efficient  to  administer  and  should  be  capable 
of  being  administered  via  computer.  The  Keleman  et  al.  (1990;  1991)  approach  does  not  have 
those  characteristics.  In  addition,  that  approach  is  designed  to  determine  whether  someone  can 
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apply  knowledge  in  a  role  play;  our  focus  for  Stage  1  is  finding  out  whether  the  Soldier  has  the 
underlying  knowledge  that  could  allow  him  or  her  to  demonstrate  the  skill 

It  is  likely  that  in  our  assessment,  we  would  present  an  examinee  with  a  computer-animated 
situation  to  which  the  examinee  would  be  asked  to  make  an  open-ended  response.  For  example, 
the  examinee  could  be  asked  to  identify  the  salient  facets  of  the  situation  (e.g.,  tension,  status 
differences  between  actors),  or  asked  what  are  likely  outcomes  given  certain  actions,  factors  to 
consider  in  deciding  how  to  respond,  or  pros  and  cons  of  various  actions  to  take.  This  would  be  a 
unique  method  of  measuring  the  knowledge  of  the  I/I  KSAs  possessed  by  an  individual.  It  can 
capture  information  that  is  different  than  that  identified  in  an  SJT,  even  though  both  tests  might 
tap  the  same  KSAs. 

We  envision  action  item  scenarios  to  be  similar  to  SJT  scenarios,  but  provide  more  explicit 
nonverbal  cues  that  would  be  relevant  to  interpreting  the  problem  situation.  To  try  out  the 
concept,  we  propose  to  take  3-4  scenarios  (perhaps  taken  from  the  Select21  or  NC021  SJT 
development  work)  and  craft  questions  that  would  pertain  to  the  KSAs  we  are  trying  to  assess. 
During  development  of  the  action  exam,  we  would  expect  to  use  a  short-answer  response  format, 
which  would  allow  us  to  identify  a  large  number  of  responses  to  each  scenario.  These  responses 
could  later  be  used  to  develop  a  set  of  multiple-choice  answers  to  be  used  in  the  primary  Army 
application  or  as  a  set  of  options  that  the  examinee  would  rank  from  best  to  worst.  The  response 
type  would  depend  on  the  specific  question  being  asked.  This  will  make  large-scale  data 
collection  and  scoring  more  efficient. 

The  action  exam  offers  the  opportunity  to  assess  knowledge  areas  that  cannot  be  easily  assessed 
with  other  measures,  and  it  offers  an  interesting  research  opportunity.  However,  there  is  little  or 
no  research  on  this  type  of  exam,  so  we  would  have  to  rely  on  our  collective  test  development 
experience  in  developing  such  a  test.  Another  possible  disadvantage  of  the  action  exam  is  that  we 
have  no  proof  that  it  would  be  an  effective  predictor  of  performance,  and  the  cost  of 
development  would  not  be  small.  The  SJT  has  a  solid  track  record  and  we  have  experience 
developing  this  type  of  test  (Keenan,  1997;  Schmitt  et  al.,  1994;  Waugh,  2002),  which  has  been 
shown  to  have  incremental  validity  over  the  ASVAB  (Campbell,  Knapp,  &  Heffner,  2002). 

KSAs  that  could  be  measured  via  an  action  exam  or  SJT  include: 

•  Conflict  Management 

•  Cultural  Tolerance 

•  Relating  to  and  Supporting  Others 

•  Team  Orientation 

•  Coordination 

•  Cooperativeness  in  Problem  Solving 

•  Leading  Others 

•  Social  Perceptiveness 

•  Ability  to  Communicate  Orally 
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Stage  2  Measures 

We  anticipate  Stage  2  assessments  will  include  a  leaderless  group  discussion  (LGD),  a  semi- 
structured  interview,  and  a  test  of  written  communication  ability. 

Leaderless  Group  Discussion 

In  an  LGD,  small  groups  of  participants  are  given  a  job-related  problem  to  solve  or  a  job-related 
issue  to  discuss.  No  leader  is  appointed,  hence  the  name  of  the  exercise.  We  propose  to  develop 
LGD  materials  and  procedures  that  would  be  administered  in  a  fairly  traditional  way  (i.e.,  not 
involving  computers).  As  the  participants  discuss  the  problem  or  issue,  their  activity  will  be 
videotaped  and  later  scored  by  trained  judges.  Typically,  participants  are  rated  on  dimensions 
such  as  cooperativeness,  leadership,  and  analytical  skills.  However,  the  way  the  exercise  is  set  up 
and  the  type  of  problem  or  issue  given  to  examinees  to  discuss  will  determine  the  KSAs  that  can 
be  measured.  In  some  ways,  this  is  quite  similar  to  developing  a  role  play.  Development  will 
include  ideas  about  how  to  frame  the  exercise  so  that  participants  are  given  the  opportunity  to 
display  their  skill  in  a  variety  of  areas.  Core  KSAs  that  could  be  assessed  using  this  method 
include: 

•  Conflict  Management 

•  Relating  to  and  Supporting  Others 

•  Cultural  Tolerance 

•  Team  Orientation 

•  Social  Perceptiveness 

•  Leading  Others 

•  Oral  Communication 

Structured  Interview 

The  interview  is  one  of  the  most  commonly  used  methods  for  selecting  employees  for  hiring, 
training,  and  promotion.  Much  of  the  early  research  investigating  the  validity  of  typical  job 
interviews  was  disappointing.  Structured  interviews,  however,  have  proven  valid  in  many 
different  contexts  (e.g..  Campion,  Pursell,  &  Brown,  1988;  Harris,  1989;  Latham,  Saari,  Pursell, 
&  Campion,  1980;  Pulakos,  Schmitt,  &  Keenan,  1994).  In  a  typical  structured  interview, 
interviewees  are  presented  a  set  of  standardized  questions  developed  to  tap  KSAs  relevant  to  the 
job.  Responses  to  the  structured  questions  are  then  evaluated  using  well-defined,  standardized 
scoring  criteria.  A  structured  interview  is  useful  in  measuring  a  variety  of  interpersonal  skills, 
which  are  often  difficult  to  assess  well  in  other  types  of  assessments.  In  addition,  the  interview 
provides  an  excellent  opportunity  to  assess  oral  communication  ability. 

Different  approaches  to  developing  structured  interview  questions  have  been  discussed  in  the 
literature.  “Experience-based”  interviews  ask  interviewees  to  describe  experiences  from  their 
past  relevant  to  the  different  dimensions  of  interest.  Advocates  of  using  experienced-based 
interview  questions  take  the  position  that  the  best  predictor  of  future  performance  is  past 
performance  on  the  same  competencies.  Previous  research  has  shown  that  experience-based 
interview  questions  are  effective  in  predicting  job  performance  for  several  different  occupations 
(e.g.,  Motowidlo  et  al.,  1992;  Pulakos  et  al.,  1994). 
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Other  researchers  have  adopted  a  slightly  different  approach,  referred  to  as  “situational,”  to 
developing  structured  interview  questions.  Situational  questions  present  interviewees  with 
hypothetical  situations  and  ask  them  to  describe  what  they  would  do  if  they  were  faced  with  the 
situation  described.  Situational  questions  have  a  couple  of  possible  advantages  over  experience- 
based  questions.  First,  because  all  interviewees  are  asked  to  respond  to  the  same  situation  rather 
than  describe  whatever  situation  they  may  wish  to  relay  from  their  past,  the  responses  are  likely 
to  be  easier  to  compare  and  hence  score  reliably  across  interviewees.  Second,  situational 
questions  allow  interviewees  to  indicate  how  they  would  respond  to  a  given  type  of  situation 
even  if  they  have  had  no  actual  past  experience  to  relay  relevant  to  the  dimension  being  asked. 

We  propose  to  adapt  the  semi-structured  interview  from  the  NC021  project  (Knapp  et  al.,  2002). 
This  interview  includes  three  types  of  interview  questions:  experience-based,  situational,  and 
fact-based.  Experience-based  questions  ask  the  respondent  to  describe  the  situation,  how  he/she 
behaved  in  the  situation,  and  the  result  of  the  action.  Hypothetical  situation  questions  present  the 
respondent  with  a  realistic  scenario  and  ask  him/her  to  describe  what  he/she  would  be  likely  do 
in  the  situation.  Fact-based  questions  assess  MOS/occupation-specific  knowledge  and  skills  and 
have  clear  right  answers.  This  may  not  be  appropriate  for  identifying  whether  a  Soldier  knows 
how  to  apply  interaction/interpersonal  KSAs,  so  we  will  talk  with  SMEs  as  a  basis  for  making 
the  decision  as  to  whether  this  type  of  question  should  be  included  in  the  semi-structured 
interview. 

The  NC021  semi-structured  interview  uses  a  standard  protocol  for  conducting  the  interview, 
selecting  questions  from  a  question  bank,  developing  new  questions,  and  evaluating  interviewees 
in  several  target  areas.  Basic  components  of  the  interview  include  (a)  a  question  bank,  (b)  target 
area  definitions,  (c)  anchored  rating  scales  for  each  of  the  target  areas,  (d)  instructions  and 
worksheet  for  developing  questions  to  supplement  the  question  bank,  and  (e)  a  worksheet  on 
which  to  record  and  consolidate  ratings  from  the  interviewers.  During  the  interview,  a  Soldier  is 
asked  a  number  of  questions  by  a  panel  of  two  or  more  senior  NCOs.  These  NCOs  then  use  the 
anchored  rating  scales  to  assess  the  Soldier’s  performance  in  the  interview.  Each  rating  scale 
ranges  from  1  (low  effectiveness)  to  7  (high  effectiveness)  and  contains  three  anchor  levels  (i.e., 
low,  moderate,  and  high).  Each  anchor  includes  a  brief  description  about  behavior  demonstrated 
at  that  level  and  several  behavioral  examples  of  what  the  interviewee  might  describe  in  his/her 
response. 

Core  KSAs  that  could  be  measured  with  a  semi-structured  interview  include: 

•  Conflict  Management 

•  Social  Perceptiveness 

•  Relating  to  and  Supporting  Others 

•  Team  Orientation 

•  Leading  Others 

•  Cultural  Tolerance 

•  Oral  Communicatioa 

Writing  Skill  Assessment 
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It  is  very  important  to  ensure  that  the  information  and  intent  of  a  written  message  are  both  clearly 
communicated.  Increasingly  people  are  communicating  via  email  and  this  is  expected  to  the  case 
for  the  future  force  Soldier  as  well.  It  is  an  easy-to-use  medium  that  allows  immediate 
communication  from  a  distance.  A  significant  danger  associated  with  this  medium  of 
communication  is  that  the  reader  often  misconstrues  messages  because  the  message  does  not 
have  the  context  present  in  a  conversation.  There  are  no  vocal  tones,  no  nonverbal  behaviors,  or 
facial  expressions  to  give  cues  about  whether  something  is  said  in  jest  or  sarcastically,  or 
whether  the  sender  really  intended  to  say  something  unpleasant.  This  is  one  of  the  reasons  that 
from  the  early  stages  of  email  development,  conventions  were  developed  to  help  put  a  voice  to 
the  written  word.  These  conventions  included  such  things  as  emoticons  (e.g.,  smiley  or  frown 
faces)  and  not  using  all  capital  letters  (which  indicates  shouting). 

We  propose  to  develop  a  test  of  written  communication  that  assesses  the  clarity  of  a  message, 
both  in  content  and  tone.  This  test  will  not  assess  technical  writing  skills  such  as  punctuation, 
grammar,  or  vocabulary.  The  test  will  likely  involve  construction  of  several  email  messages  in 
response  to  a  stated  problem.  There  might  be  a  single  situation  that  requires  several  messages 
and/or  several  situations.  We  will  try  out  different  approaches  to  setting  up  the  exam,  including 
informing  examinees  about  how  their  messages  will  be  scored  (e.g.,  being  able  to  get  the 
message  across  is  more  important  than  grammar  or  spelling)  versus  not  providing  this 
information  and  providing  more  or  less  guidance  on  the  parameters  of  the  messages  they  are 
expected  to  craft.  It  will  also  be  interesting  to  look  at  whether  we  can  get  more  informative 
assessments  by  giving  examinees  strict  time  limits  versus  allowing  them  ample  time  to  construct 
and  review  their  messages. 

Unlike  the  other  Stage  2  assessments,  the  writing  examination  will  be  entirely  automated.  What 
is  unclear  at  this  stage,  however,  is  whether  we  can  develop  an  automated  scoring  routine.  At  this 
point,  we  think  it  is  likely  that  some  human  judgment  will  be  required  for  scoring  the  test,  but 
that  these  judgments  will  be  very  straightforward  to  make. 

Effects  of  Uncontrolled  Variables 


In  addition  to  the  1/1 KSA  assessments,  we  recommend  including  measures  of  general  cognitive 
aptitude  and  personality  in  the  Phase  II  research.  As  explained  in  this  section,  these  measures 
will  help  ensure  a  fuller  understanding  of  how  these  variables  impact  how  well  examinees 
perform  in  the  assessment  battery. 

Each  of  the  I/I  KSAs  could  be  assessed  using  observations  of  actual  job  performance.  However, 
it  is  important  to  remember  that  knowing  what  to  do  and  having  the  skill  to  use  the  knowledge 
may  not  always  result  in  the  expected  behavior.  The  difference  between  actual  performance  and 
skill  as  assessed  by  tests  is  that  the  performance  context  adds  additional  sources  of  variation  that 
are  controlled  for  in  a  skill  assessment  [personal  communication,  John  Campbell,  2003]. 

Variance  in  skill  level  as  assessed  in  a  standardized  measurement  procedure  is,  in  turn,  a  function 
of  general  cognitive  aptitude,  procedural  knowledge  relevant  for  the  skill,  and  a  variety  of 
dispositional  variables  (e.g.,  personality)  that  are  viewed  as  stable  traits.  Dispositional  variables 
that  are  not  stable  would  be  things  like  self-efficacy,  which  is  likely  to  vary  according  to  the 
situation.  Personality  dispositions  play  a  role  because  we  are  interested  in  assessing  interpersonal 
skills  that  may  be  constrained  or  enhanced  by  one’s  personality.  The  same  would  not  be  true  for 
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standardized  assessments  of  technical  skills.  One  could  “know”  what  to  do  relative  to  skill 
expression,  but  have  difficulty  doing  it,  even  in  a  role  play,  because  of  constraints  imposed  by 
one’s  “personality.”  Figure  4  presents  a  diagram  of  the  way  we  view  the  interaction  of  general 
cognitive  aptitude,  personality,  knowledge,  and  skill  on  performance.  The  solid  lines  are  the 
hypothesized  direct  effects.  The  dashed  lines  are  residual  direct  effects  that  could  occur.  For 
example,  general  cognitive  aptitude  could  have  a  residual  effect  on  skill  level,  or  on 
performance,  even  after  its  direct  effect  on  knowledge  is  accounted  for. 

Anticipating  the  importance  of  personality  and  general  cognitive  aptitude  A  in  performance  on 
the  experimental  assessment  battery,  we  propose  to  include  personality  measures  in  the  Stage  1 
assessment  package  and  will  retrieve  an  estimate  of  general  cognitive  aptitude  from  Soldiers’ 
automated  personnel  records.  Specifically,  in  addition  to  the  TAP/BIQ  Amalgam  (a  biodata 
measure  targeted  to  assess  personality  variables),  we  will  include  a  measure  derived  from  the 
International  Personality  Item  Pool  (IPIP)  (currently  being  used  as  a  marker  test  for  personality 
for  Select21)  and  possibly  the  Person-Organization-Personality  (POP)  Hybrid,  a  personality 
measure  being  developed  in  the  Select21  project.  The  general  cognitive  aptitude  measure  will 
either  be  the  General  Technical  or  Armed  Forces  Qualification  Test  scores  from  the  Armed 
Services  Vocational  Aptitude  Battery  (ASVAB). 


L.C..T: 


Figure  4.  The  hypothesized  effects  of  general  cognitive  aptitude,  personality,  knowledge,  and 
skill  on  performance. 


Personality  Measures 


International  Personality  Item  Pool  (IPIP) 

The  International  Personality  Item  Pool  (IPIP)  bills  itself  as  “A  Scientific  Collaboratory  for  the 
Development  of  Advanced  Measures  of  Personality  Traits  and  Other  Individual  Differences” 
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(http://ipip.ori.org/ipip/).  A  collaboratory  is  a  computer-supported  system  that  allows  scientists  to 
work  with  each  other,  facilities,  and  data  bases  without  regard  to  geographical  location  (Finholt  & 
Olson,  1997).  The  IPIP  Website  is  intended  to  provide  rapid  access  to  measures  of  individual 
differences,  all  in  the  public  domain,  to  be  developed  conjointly  among  scientists  worldwide.  The 
site  provides  a  pool  of  1,956  items  organized  into  280  scales  that  have  been  developed  to  measure 
constructs  similar  to  those  in  proprietary  tests,  such  as  the  NEO-PI  (Costa  &  McRae,  1992).  The 
relevant  IPIP  scales  line  up  against  the  Big  Five  personality  factors  as  follows  (see  Table  5): 

Table  5.  Comparison  ofBig  Five  Personality  Factory  to  IPIP  Scales _ 

Big  Five  Personality  Factors  IPIP  Scales 

•  Conscientiousness  •  Dependability,  Need  to  Achieve,  Self-Reliance 

•  Agreeableness  •  Agreeableness  (Relating  to  and  Supporting  Others) 

•  Neuroticism  •  Emotional  Stability 

•  Openness  to  Experience  •  Intellectance  (Adaptability/Flexibility) 

•  Extroversion  •  Potency,  Affiliation  (Team  Orientation) 

Any  researcher  who  wishes  to  develop  an  instrument  is  free  to  use  the  items  in  the  IPIP.  The 
website  offers  many  options.  For  example,  there  is  set  of  50  items  that  are  balanced  between 
positively  and  negatively  keyed  responses  and  ready  to  be  used.  This  set  of  items  is  being  tested 
in  the  Select2 1  project  as  a  marker  for  other  measures  related  to  personality  or  temperament.  A 
marker  test  is  one  that  has  been  shown  to  measure  quite  well  one  or  more  particular  personality 
characteristics  (for  example,  agreeableness).  We  use  a  marker  test  as  a  basis  of  comparison  when 
trying  out  a  new  or  experimental  measure  designed  to  assess  the  same  characteristic(s).  We 
compare  examinees’  scores  on  the  marker  test  to  scores  on  the  experimental  measure.  If  the 
experimental  measure  works  as  it  is  supposed  to  (that  is,  if  it  measures  the  characteristic(s)  well), 
then  people  who  score  high  (low)  on  the  marker  test  should  also  score  high  (low)  on  the 
experimental  measure-that  is,  they  should  score  the  same  on  both  measures.  Researchers  who 
use  the  scales  often  report  their  results  to  the  host  of  the  site  (Goldberg,  1999),  who  updates  the 
statistics.  The  Agreeableness  scale  (equivalent  to  our  Amicability  KSA)  has  an  alpha  coefficient 
of  .77.  The  items  in  that  scale  are  shown  in  Figure  5. 


Positively  keyed  items 

Have  a  good  word  for  everyone. 

Believe  that  others  have  good  intentions. 

Respect  others. 

Accept  people  as  they  are. 

Make  people  feel  at  ease. 

Negatively  keyed  items 

Have  a  sharp  tongue. 

Cut  others  to  pieces. 

Suspect  hidden  motives  in  others. 

Get  back  at  others. 

Insult  people. 

Figure  5.  Positively  and  negatively  keyed  items  from  the  IPIP  Agreeableness  scale. 


The  site  provides  a  set  of  sample  instructions  and  response  options  for  administering  IPIP  items. 
They  read  as  follows: 

“On  the  following  pages,  there  are  phrases  describing  people's  behaviors.  Please  use  the 
rating  scale  below  to  describe  how  accurately  each  statement  describes  you.  Describe 
yourself  as  you  generally  are  now,  not  as  you  wish  to  be  in  the  future.  Describe  yourself  as 
you  honestly  see  yourself,  in  relation  to  other  people  you  know  of  the  same  sex  as  you  are, 
and  roughly  your  same  age.  So  that  you  can  describe  yourself  in  an  honest  manner,  your 
responses  will  be  kept  in  absolute  confidence.  Please  read  each  statement  carefully,  and  then 
fill  in  the  bubble  that  corresponds  to  the  number  on  the  scale.” 

Response  Options 

1 :  Very  Inaccurate 
2:  Moderately  Inaccurate 
3:  Neither  Inaccurate  nor  Accurate 
4:  Moderately  Accurate 
5:  Very  Accurate 

Because  we  had  established  the  advisability  of  assessing  personality  factors,  we  then  were  faced 
with  the  task  of  identifying  a  measure  that  would  fit  into  our  project  needs  and  budget.  We  had 
previously  decided  that  using  a  COTS  product  was  not  feasible  due  to  cost  and  the  time  lapse  in 
scoring  the  tests.  The  EPIP  appears  to  offer  a  feasible  solution  to  these  constraints.  According  to 
the  information  on  the  website,  the  items  are  free  to  use  with  no  permission  required,  they  have 
been  used  in  many  studies  (citations  provided  on  the  website),  the  existing  scales  have  internal 
reliability  statistics,  and  the  items  have  been  shown  to  correlate  with  the  NEO  factors.  We  have 
the  flexibility  of  creating  our  own  scales  should  we  wish  to  do  so  or  using  either  the  50-  or  100- 
item  scales  that  have  already  been  developed. 

Person-Organization-Personality  (POP)  Hybrid 

The  Person-Organization-Personality  (POP)  Hybrid  is  a  self-report  personality  measure  being 
developed  for  Select21.  It  is  designed  to  assess  how  well  the  person  and  organization  fit  together 
and  whether  the  organization  provides  work  that  has  characteristics  the  individual  finds 
enjoyable.  Using  either  a  forced  choice  or  ranking  format  (the  approach  is  still  being  developed) 
respondents  indicate  work  requirements  at  which  they  think  they  would  be  most  successful.  Two 
sample  POP  Hybrid  items  are  listed  below: 

Work  that  requires. . . 

A.  Showing  a  cooperative  and  friendly  attitude  towards  others  you  dislike  or  disagree  with 

B.  Being  open  to  change  (positive  or  negative)  and  a  lot  of  variety 

As  noted  previously,  a  commonly  acknowledged  problem  with  personality  measures  is  that  people 
can  fake  their  responses  to  make  themselves  look  better  (Hough,  Eaton,  Dunnette,  Kamp  &  McCloy, 
1990;  Ones,  Viswesvaran,  &  Korbin,  1995)  and  that  many  will  do  so  in  operational  selection  settings 
(Hough,  1996, 1997, 1998;  Rosse,  Stechler,  Miller,  &  Levin,  1998).  Given  their  experience  with  the 
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Army’s  Assessment  of  Individual  Motivation  (AIM;  Knapp,  Waters,  &  Heggestad,  2002), 
researchers  on  the  Select21  project  believe  that  response  distortion  poses  a  dauntingly  high  hurdle  to 
the  personnel  selection  specialist  interested  in  using  temperament  measures  in  an  operational  setting. 

Recent  efforts  to  reduce  response  distortion  have  centered  on  forced-choice  formats,  which  ask 
the  respondent  to  choose  between  two  alternatives  as  being  more  appealing  to  him/her.  Each 
alternative  in  the  instrument  is  paired  with  every  other  alternative.  This  approach  has 
demonstrated  the  capacity  to  reduce  the  effects  of  faking  (Jackson,  Wrobleski,  &  Ashton,  2000; 
White  &  Young,  1998;  Wright  &  Miederhoff,  1999),  but  it  is  criticized  because  it  results  in 
ipsative  response  data.  It  is  too  often  easy  for  respondents  to  identify  the  constructs  of  interest 
and  to  skew  their  responses  to  make  themselves  look  good. 

The  15  statements  in  the  POP  Hybrid  are  adapted  from  the  Work  Styles  portion  of  the  0*NET 
content  model  (Borman,  Kubisiak,  &  Schneider,  1999).  The  wording  has  been  simplified  to 
make  them  more  accessible  to  a  casual  reader.  KSAs  that  can  be  measured  with  the  POP  Hybrid 
include: 

•  Dependability 

•  Relating  to  and  Supporting  Others 

•  Social  Perceptiveness 

•  Team  Orientation 

We  believe  this  combination  of  measures  will  allow  us  to  determine  whether  examinees  have 
sufficient  knowledge  about  what  to  do  in  various  situations,  or  how  to  read  situations,  so  that 
they  could  perform  well  in  a  standardized  environment.  The  collection  of  biodata  and  personality 
data  will  help  us  control  for  the  effects  of  those  variables  in  our  analysis. 

Experience  Measures 

One  of  the  best  predictors  of  knowledge  level  will  be  experience.  Each  participant  will  provide 
his  or  her  pay  grade,  which  will  be  a  good  estimate  of  experience.  As  an  additional  measure  of 
experience,  we  will  develop  an  instrument  similar  to  the  Experience  and  Activities  Record 
(ExAct)  developed  for  NC021.  The  ExAct  is  a  46-item  self-report  measure  designed  to  “assess 
the  extent  to  which  a  Soldier  has  engaged  in  specific  activities  or  had  particular  experiences  that 
may  predict  performance  at  the  next  grade”  (Knapp  et  al.,  2002,  p.  55).  The  premise  behind  use 
of  such  an  instrument  is  that  having  prior  experience  should  prepare  a  Soldier  to  perform  those  or 
similar  activities  in  the  future.  The  ExAct  itself  is  not  likely  to  be  suitable  for  the  more  limited 
set  of  I/I  KSAs  in  which  we  are  interested,  but  it  will  provide  a  model  we  can  use  to  guide  our 
thinking  about  what  types  of  experience  would  be  appropriate  to  consider  for  this  project. 

Computerization  of  Proposed  Assessment  Methods 

Traditionally,  measures  of  I/I  KSAs  have  been  administered  either  as  paper-and-pencil 
instruments  (e.g.,  questionnaires,  tests)  or  as  interviews  or  other  tests  that  were  manually  scored 
by  observers.  Many  of  these  assessments  have  answer  formats  that  allow  them  to  be  scored  by 
computers,  with  participant  information  and  scores  stored  in  a  database.  More  recently,  some  of 
these  measures  have  been  computerized  so  that  the  participant  reads  the  test  items  on  the 
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computer  interface  and  responds  using  the  keyboard.  But  this  assessment  technique  does  not  take 
advantage  of  new  technological  and  bandwidth  capabilities  readily  available  on  laptops  and  other 
portable  devices  such  as  Personal  Digital  Assistants  (PDAs). 

Current  advanced  information  technology  and  the  increased  bandwidth  available  on  laptop 
computers  and  other  portable  devices  has  allowed  computerized  versions  of  assessment 
instruments  to  be  developed  that  can  be  administered  in  almost  any  setting  by  personnel  without 
specialized  training  in  test  administration.  Micro  Analysis  and  Design  (MA&D)  has  developed 
several  innovative  computer-based  training  tools  that  have  tests  as  a  component.  Two  examples 
of  these  tools,  the  Course  of  Action  Training  Tool  (COATT)  and  the  Multimedia  Instructional 
Tutoring  Authoring  System  (MIT AS),  could  serve  as  the  basis  for  innovative  I/I  KSA 
measurement  techniques.  MIT  AS  and  COATT,  which  are  interactive  and  incorporate 
multimedia,  3D  graphical  animation,  speech  recognition,  natural  language  processing,  exercise 
development  templates,  and  performance  based  sequencing,  offer  us  the  opportunity  to  create 
valid  and  reliable  versions  for  computer-based  action  tests  that  are  easily  administered  and 
scored.  They  also  collect  performance  data  and  provide  performance  feedback. 

In  Stage  1  of  the  I/I  KSA  assessment  process,  we  intend  to  leverage  the  technologies  that  are  part 
of  COATT  and  MIT  AS  to  aid  in  the  development  of  the  action  exam  or  SJT.  The  technologies 
employed  in  these  two  tools  have  a  variety  of  potential  applications  for  the  development  of  the 
action  exam  or  SJT.  One  potential  KSA  that  can  be  measured  by  the  action  exam  or  SJT  is 
conflict  management.  Assessing  this  KSA  in  the  action  exam  or  SJT  using  COATT  and  MIT  AS 
technologies  could  take  on  a  number  of  different  forms. 

For  example,  COATT  utilizes  a  task  network  model  of  the  process  of  piloting  a  helicopter  to  an 
accident  scene.  The  model  simulates  the  actions  the  pilot  would  take,  such  as  requesting  weather 
updates,  communicating  current  position,  changing  altitude,  and  deciding  whether  to  continue 
with  the  mission.  As  the  model  executes,  a  pilot  in  training  watches  as  rolling  video  depicting  the 
given  scenario  is  played  on  the  screen  (see  Figure  6).  At  critical  decision  points  in  the  model,  the 
trainee  is  asked  to  make  decisions  about  the  most  appropriate  actions  to  take.  In  COATT,  the 
scenario  responds  to  the  pilot’s  actions  and  can  take  different  paths  based  on  the  pilot’s  choices. 
For  our  purposes,  the  use  of  branching  would  be  limited  because  branching  significantly 
complicates  scoring.  However,  the  ability  of  the  COATT  tool  to  depict  relevant  scenarios  using 
live  action  video  footage,  to  halt  that  footage  at  critical  decision  points  and  request  a  response 
from  the  participant  could  be  implemented  for  assessing  the  I/I  KSAs. 
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Figure  6.  The  course  of  action  training  tool. 


The  technique  could  be  applied  to  assessing  a  Soldier’s  ability  to  manage  an  interpersonal  conflict 
situation  using  either  rolling  video  or  3-D  computer  simulations  (see  Figure  7).  In  this  context,  the 
underlying  model  would  represent  the  process  of  working  through  the  interpersonal  conflict 
situation  and  managing  the  scenario  to  successfully  resolve  conflict.  As  the  task  network  model  of 
the  conflict  scenario  executed,  the  video  of  a  scenario  or  the  3-D  animation  could  play.  When  the 
model  reached  critical  decision  points  the  Soldier  could  be  asked  to  either  choose  the  best  response 
given  the  conflict  they  have  been  observing,  or  the  Soldier  could  be  required  to  do  things  such  as 
identify  key  characteristics  of  the  scenario,  provide  a  situation  status  report  to  their  supervisor  or  to 
specify  the  action  they  would  take  to  lead  to  resolution  of  the  conflict.  However,  instead  of 
choosing  the  path  to  follow  based  on  the  Soldier’s  decision,  the  assessment  would  record  and  score 
the  candidate’s  response  and  continue  along  the  pre-defmed  scenario  path.  As  the  scenario 
progressed  at  each  pre-determined  point  the  scenario  could  be  stopped  and  the  user  prompted  for  a 
response.  For  a  more  traditional  SJT,  each  scenario  would  have  a  set  of  response  options  associated 
with  it.  In  contrast,  in  an  action  exam  variation,  the  scenario  could  be  stopped  at  key  points  and  a 
set  of  questions  presented  and,  after  capturing  the  responses,  the  same  scenario  could  continue 
allowing  further  observation  of  the  developing  scenario.  The  responses  would  be  captured  and 
stored  for  use  in  scoring  the  user’s  assessment  and  for  the  purpose  of  collecting  data  on  individual 
item  properties  and  the  properties  of  the  test  as  a  whole.  By  administering  the  test  via  a  computer 
and  storing  data  on  examinee  performance,  the  assessment  tool  would  provide  information 
regarding  the  soundness  of  the  test,  thus  allowing  for  frequent  evaluation  of  the  assessment.  Along 
with  presenting  the  scenario  using  video  footage  or  computer  simulation,  COATT  could  provide 
the  capability  to  present  response  options  through  video  or  simulation.  In  this  case,  following  a 
scenario  stem  a  set  of  alternate  endings  could  be  played  out  for  the  examinee  and  they  could 
choose  the  most  acceptable  outcome  of  the  scenario.  It  is  unlikely  that  we  will  employ  this  option 
because  the  choice  of  a  correct  response  would  require  a  significant  amount  of  memory. 
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A  second  possible  implementation  of  the  technologies  in  CO  ATT  and  MIT  AS  to  assessing  I/I 
KSAs  is  the  use  of  MITAS’s  built  in  exercise  templates  to  aid  in  rapid  test  development, 
prototyping  and  update  (Figure  8).  The  MITAS  software  suite  offers  a  set  of  exercise  development 
templates  that  allow  the  test  developer  to  associate  video,  audio,  or  text  materials  with  a  given 
item.  For  an  action  exam  or  SJT,  these  templates  could  be  used  to  create  a  wide  variety  of  items, 
including  multiple-choice,  short  answer,  and  rank  ordering  of  response  alternatives.  For  each  of 
these  exercises  a  specific  scenario  could  be  presented  either  through  text  as  in  traditional  versions 
of  these  assessments  or  through  video  or  computer  animation.  Additionally,  the  MITAS  tool  allows 
for  easy  sequencing  of  items  through  a  drag  and  drop  interface.  Using  the  MITAS  interface  would 
enable  testing  of  multiple  test  configurations  during  the  test  development  process  to  explore  order 
effects  that  may  influence  the  difficulty  of  specific  test  items. 


Figure  8.  Exercise  development  template  in  MITAS. 

In  Stage  1  of  the  assessment  process,  the  ability  to  present  customized  scenarios,  using  either  live 
action  video  footage  or  computer  simulation,  could  be  used  to  present  the  stimuli  in  a  structured 
interview.  In  this  setting,  computer  technology  would  be  employed  to  present  the  interview 
questions  through  video  or  computer  simulation.  The  videotaped  interviewer  or  a  simulated 
interviewer  could  be  designed  to  ask  questions  of  the  examinee.  The  participant  could  then 
respond  to  the  questions  either  verbally  and  have  the  responses  captured  using  either  video  or 
audio  tape  (or  both)  or  through  a  text  based  response  entered  via  a  keyboard.  The  responses 
would  then  be  stored  for  later  evaluation  by  trained  assessors.  The  use  of  technology  to  present 


27 


the  interview  questions  eliminates  the  need  for  a  trained  interviewer  and  as  such  requires  less  up 
front  resources  to  administer  the  assessment.  Additionally,  if  responses  were  recorded  textually 
via  a  keyboard,  as  technologies  geared  toward  interpreting  meaning  from  text  (such  as  Latent 
Semantic  Analysis)  mature,  they  could  be  implemented  in  the  assessment  to  provide  automated 
scoring  of  the  structured  interview. 

The  technologies  that  underlie  the  CO  ATT  and  MITAS  products  offer  a  variety  of  opportunities 
for  assessing  I/I  KS  As.  From  the  ability  to  present  realistic  scenarios  using  video  footage  or 
computer  animation  to  capturing,  scoring  and  storing  detailed  information  on  examinee 
performance  and  rapid  prototype  development,  the  use  of  emerging  technologies  will  create  a 
powerful  tool  for  assessing  I/I  KSAs. 


PHASE  n  PLAN 

The  Phase  II  effort  will  follow  a  traditional  assessment  development  approach  to  create  the  2- 
stage  measurement  battery.  With  a  projected  start  in  November  of  2003,  the  project  will  begin  by 
drafting  the  scenarios  for  the  action  exam  that  will  be  included  in  the  first  stage  of  the  assessment 
process.  Along  with  developing  the  action  exam  scenarios,  development  of  the  structured 
interview  and  LGD  materials  will  begin.  In  the  early  stages  of  developing  the  action  exam 
scenarios,  interview  and  LGD  materials,  project  staff  will  present  the  materials  to  SMEs  in  an 
initial  workshop.  During  the  workshop  the  SMEs  will  comment  on  the  scenarios,  help  flesh  them 
out,  make  them  appropriate  for  use  with  the  Army  and  provide  ideas  for  additional  scenarios.  At 
subsequent  workshops,  SMEs  will  review  progress  made  on  the  materials  and  continue  to 
provide  feedback  to  shape  the  materials  into  useful  assessments  targeted  for  the  Army.  Once  the 
initial  draft  materials  are  prepared,  another  SME  workshop  will  be  convened  where  SMEs  will 
identify  possible  actions  that  the  respondent  could  take.  This  list  of  actions  will  be  reviewed  in  at 
least  two  workshops  and  the  output  of  these  workshops  will  be  used  to  develop  a  scoring 
protocol. 

Along  with  developing  the  test  materials  we  plan  to  establish  construct  and  criterion-related 
validity  of  the  predictor  assessments.  To  do  this  we  will  work  with  ARI  to  identify  existing 
performance  measures  that  may  be  appropriate  criteria  for  the  current  effort.  These  criterion 
measures  may  include  job  performance  ratings  scales  developed  specifically  for  this  project  as 
well  as  existing  criteria  such  as  relevant  scales  from  the  NC021  and  Select21  criterion  rating 
scales.  Each  of  the  criteria  will  be  used  in  a  criterion-related  validation  effort  once  the  final  VI 
KSA  measures  have  been  developed. 

Once  the  initial  draft  assessments  have  been  developed  and  appropriate  criterion  measures  have 
been  selected  we  will  conduct  internal  reviews  with  contractor  staff  members  and  ARI  personnel 
to  serve  as  pre-test  examinees.  The  assessments  will  then  be  revised  and  finalized  in  preparation 
for  pilot  testing  with  50-75  Soldiers  at  one  or  two  Army  installations  in  April  2005.  The  pilot 
testing  will  include  collection  of  performance  measures  used  to  provide  initial  evidence  of 
instrument  validity.  Following  the  pilot  testing,  the  contractor  team  will  evaluate  the  assessments 
for  their  psychometric  properties  and  revise  them  as  required  to  maximize  their  reliability  and 
validity. 
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The  final  step  in  the  assessment  development  process  will  be  to  conduct  the  larger  scale 
validation  effort.  Ideally,  this  validation  effort  would  include  assessing  at  least  200  Soldiers 
representing  various  MOS  across  four  to  five  Army  installations1.  We  also  intend  to  collect 
criterion  ratings,  which  will  require  access  to  supervisors  and/or  peers  of  the  Soldier  participants 
to  provide  performance  ratings.  Along  with  collecting  data  on  the  VI KSA  measures,  we  intend 
to  collect  data  on  participants’  general  cognitive  aptitude  as  measured  by  the  ASVAB.  The  GMA 
measure  will  allow  us  to  determine  the  relationship  between  our  measures  of  the  I/I  KSAs  and 
overall  GMA  and  account  for  this  relationship  in  the  scoring  routines. 

At  the  conclusion  of  Phase  II  the  contractor  team  will  provide  ARI  with  a  comprehensive  report 
detailing  the  development  activities  and  providing  commercial  quality  user  documentation  for 
the  assessment  battery.  This  report  and  accompanying  documentation  will  serve  as  the  final 
deliverable  for  the  Phase  II  effort  and  will  accompany  the  2-stage  assessment  battery  as  the 
ultimate  products  of  the  research  and  development  effort. 


1  If  Soldiers  are  not  available,  we  will  try  to  gain  access  to  college  students.  If  we  use  civilians,  we  will  need  an 
alternative  measure  of  GMA  (e.g.,  self-report  SAT  scores). 
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APPENDIX  A:  MEASUREMENT  METHODS  AND  TESTS  GROUPED  BY  KSA 


Ability  to  Relate  to  and  Support  Peers 

❖  Self-report  Method 

>  Fixed  Response 

■  California  Psychological  Inventory 

■  Personality  Research  Form 

■  Adjective  Checklist 

■  Survey  of  Interpersonal  Values 
^  Scenario-based  Fixed  Response 

■  Situational  Judgment  Test 

❖  Computer-based  Simulation  Method 

>  High  Fidelity  Stimulus/Low  Fidelity  Response 

■  Situational  Judgment  Test 


Affiliation 

❖  Self-report  Method 
^  Fixed  Response 

■  NEO-PI 

■  Edwards  Personal  Preference  Schedule 

■  Personality  Research  Form 

■  Adjective  Checklist 

■  Comfrey  personality  Scales 

*  California  Psychological  Inventory 
^  Scenario-based  Fixed  Response 

■  Situational  Judgment  Test 

❖  Computer-based  Simulation  Method 

>  High  Fidelity  Stimulus/Low  Fidelity  Response 

■  Situational  Judgment  Test 

Agreeableness 

❖  Self-report  Method 

>  Fixed  Response 

■  NEO-PI 

■  16PF 

■  Gordon  Personal  Profile 

■  Biographical  Information  Questionnaire 

■  Jackson  Personality  Inventory-Revised 

■  ABLE 

>  Forced  Choice 

■  Assessment  of  Individual  Motivation  (AIM) 
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Concern  for  Soldier  Quality  of  Life 

❖  Self-report  Method 

>  Fixed  Response 

■  California  Psychological  Inventory 

■  Personality  Research  Form 

■  Edwards  Personal  Preference  Schedule 

■  Survey  of  Interpersonal  Values 

■  Jackson  Personality  Inventory  -  Revised 

>  Scenario-based  Fixed  Response 

■  Situational  Judgment  Test 

❖  Computer-based  Simulation  Method 

>  High  Fidelity  Stimulus/Low  Fidelity  Response 

■  Situational  Judgment  Test 

Conflict  Management 

❖  Self-report  Method 

>  Fixed  Response 

■  California  Psychological  Inventory 

■  Personality  Research  Form 

■  Edwards  Personal  Preference  Schedule 

■  ABLE 

■  16PF 

>  Free  Response  -  Open-ended  Items 

■  Biodata 

>  Forced  Choice 

■  Assessment  of  Individual  Motivation  (AIM) 

>  Scenario-based  Fixed  Response 

■  Situational  Judgment  Test 

❖  Computer-based  Simulation  Method 

>  High  Fidelity  Stimulus/Low  Fidelity  Response 

■  Situational  Judgment  Test 

Cultural  Tolerance 

❖  Self-report  Method 

>  Fixed  Response 

■  Cultural  Tolerance  Scale 

■  Adjective  Checklist 

■  California  Psychological  Inventory 

■  Jackson  Personality  Inventory  -  Revised 

■  16PF 

>  Scenario-based  Fixed  Response 

■  Situational  Judgment  Test 

❖  Computer-based  Simulation  Method 

>  High  Fidelity  Stimulus/Low  Fidelity  Response 

■  Situational  Judgment  Test 

❖  Real  Life  Behavior  Method 

■  ABLE 
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Dependability 

❖  Self-report  Method 

>  Fixed  Response 

■  California  Psychological  Inventory  (CPI) 

■  16PF 

■  Gordon  Personal  Profile 

■  Personality  Research  Form 

■  Adjective  Checklist 

■  California  Psychological  Inventory 

■  Jackson  Personality  Inventory  -  Revised 

■  16PF 

>  Free  Response  -  Open-ended  Items 

■  Biodata 

>  Forced  Choice 

■  Assessment  of  Individual  Motivation  (AIM) 

❖  Computer-based  Simulation  Method 

>  High  Fidelity  Stimulus/Low  Fidelity  Response 

■  Situational  Judgment  Test 

❖  Real  Life  Behavior  Method 

■  Attendance  records,  performance  goals,  accomplishments 

Team  Orientation 

❖  Self-report  Method 

>  Fixed  Response 

■  16PF 

Adaptability/Flexibility 

❖  Self-report  Method 

>  Fixed  Response 

■  California  Psychological  Inventory 

■  16PF 

■  Biographical  Information  Questionnaire 

■  Cross  Cultural 

■  Adjective  Checklist 

■  Adaptability  Test 

Cooperativeness  in  Problem  Solving 

❖  Self-report  Method 

>  Fixed  Response 

■  California  Psychological  Inventory 

■  ABLE 

■  Adjective  Checklist 

>  Free  Response  -  Open-ended  Items 

■  Biodata 

>  Forced  Choice 

■  Assessment  of  Individual  Motivation 
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❖  Live  Action  Method 

>  Role  Play 

■  Interaction  Exam 

■  Leaderless  Group  Discussion 

Cooperation 

❖  Self-report  Method 

>  Fixed  Response 

■  Jackson  Personality  Inventory  -  Revised 

■  Adjective  Checklist 

Emotional  Intelligence 

❖  Self-report  Method 

>  Fixed  Response 

■  Balanced  Emotional  Empathy  Test 
-  16PF 

Social  Perceptiveness 

❖  Self-report  Method 

>  Fixed  Response 

■  California  Psychological  Inventory 

■  16PF 

■  Adjective  Checklist 

■  Jackson  Personality  Inventory  -  Revised 

Active  Listening 

❖  Self-report  Method 

>  Fixed  Response 

■  Biographical  Information  Questionnaire 

❖  Computer-based  Simulation  Method 

>  High  Fidelity  Stimulus/Low  Fidelity  Response 

■  Watson  Barker  HS  Listening  Test 

■  Active  Listening  Test 

❖  Live  Action  Method 

>  Individual 

■  Active  Listening  Test 

>  Role  Play 

■  Interaction  Exam 

❖  Real  Life  Behavior  Method 

>  Performance  Ratings 

■  Leaderless  Group  Discussion 

Nonverbal  Skills 

❖  Self-report  Method 

>  Fixed  Response 

■  16PF 
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❖  Live  Action  Method 

>  Role  Play 

■  Interaction  Exam 

Self-Awareness 

❖  Self-report  Method 

>  Fixed  Response 

■  Adjective  Checklist 

Oral  Communication  Skills 

❖  Self-report  Method 

>  Fixed  Response 

-  PFF21 

❖  Computer-based  Simulation  Method 

>  High  Fidelity  Stimulus/Low  Fidelity  Response 

■  The  English  Language  Skills  Profile 

❖  Live  Action  Method 

>  Role  Play 

■  Interaction  Exam 

>  Group  Simulation 

■  Alberta  Education  Oral  Communication  Evaluation 

❖  Real  Life  Behavior  Method 

>  Performance  Ratings 

■  Communication  Competency  Assessment  Instrument 

Written  Communication  Skills 

❖  Self-report  Method 

>  Fixed  Response 

■  PPF21 

Act  as  a  Role  Model 

❖  Self-report  Method 

>  Fixed  Response 

■  California  Psychological  Inventory 

Task  Leadership 

❖  Self-report  Method 

>  Fixed  Response 

■  16PF 

■  Survey  of  Interpersonal  Values 

■  Biographical  Information  Questionnaire 

Helping  Others 

❖  Self-report  Method 

>  Fixed  Response 

■  Personality  Research  Form 

■  Adjective  Checklist 

-  16PF 

■  Edwards  Personal  Preference  Schedule 
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APPENDIX  B:  UTILITY  TO  THE  ARMY  CALCULATIONS  AND  JUSTIFICATIONS 

Appendix  B  presents  the  Utility  to  the  Army  (UA)  calculations  for  each  of  the  I/I  KSAs  included 
in  the  final  taxonomy  developed  under  the  Phase  I  research  and  development  effort.  Where 
appropriate  the  KSAs  are  grouped  into  the  major  constructs  for  which  they  are  constituents.  The 
constructs,  KSAs  and  measurement  methods  described  in  this  appendix  represent  an  intermediate 
step  in  developing  the  final  I/I  KSA  taxonomy.  The  UA  scores,  coupled  with  further  research 
into  the  I/I  KSAs  led  to  a  refinement  of  the  taxonomy  that  is  reflected  in  the  discussion  of  the 
taxonomy  throughout  much  of  this  report.  In  this  appendix  the  20  I/I  KSAs  are  arranged  into 
four  major  constructs:  (l)Building  and  maintaining  relationships;  (2)Social  intelligence; 
(3)Communications  and  (4)Team  Leadership.  The  final  taxonomy  takes  the  20  individual  I/I 
KSAs  and  derives  9  core  I/I  KSAs  which  include  the  essential  features  of  the  original  list  of 
KSAs  (See  Table  2). 


Deriving  UA  Scores 

Utility  to  the  Army  =  Appropriateness  ((6-Fakability)  +  (6  -  Ease  of  Implementation)), 

or 

UA  =  A  ((6-F)  +  (6-EI)) 

If  a  measure  has  been  rated  as  “Not  appropriate”  (A  =  0)  for  assessing  a  particular  KSA,  it  essentially 
has  no  utility  to  the  Army,  in  which  case  the  UA  =  0  and  the  method  will  not  be  considered. 

UA  =  0 


If  a  measure  has  been  rated  as  “ Appropriate  ”  (A  =  1),  then  the  UA  of  that  method  is  the  sum  of  6  -  the 
Fakability  score  plus  6  -  the  Ease  of  Implementation  score  (this  reverses  the  scoring  for  these  factors  to 
make  them  directionally  the  same  as  Appropriateness).  These  scores  can  range  from  2-10.  High  scores 
have  more  UA. 

If  a  measure  has  been  rated  as  “Very  appropriate”  (A  =  2),  then  the  UA  equals  2  ((6  -  F)  +  (6  -  El)). 
These  scores  can  range  from  4  -  20.  High  scores  have  more  UA. 

The  following  pages  outline  the  UA  for  each  KSA  and  method  pairing  as  described  in  the  paragraphs 
above.  These  pages  also  provide  a  rationale  for  the  ratings.  The  ratings  and  associated  rationale  formed 
the  basis  for  the  decision  of  which  methods  to  implement  in  the  final  assessment  battery. 

Group  1:  Building  and  Maintaining  Relationships  Construct 

Rationale  for  method  selection:  These  KSAs  assess  the  Soldier’s  attitudes  toward  colleagues  and 
teammates  based  on  behaviors,  and  their  ability  to  form  and  maintain  productive  relationships.  Asking 
Soldiers  to  perform  a  task  in  a  simulation  or  role  play  that  requires  that  the  Soldier  to  demonstrate  the 
behaviors  that  are  markers  for  the  construct  Build  and  Maintain  Relationships  ( relate  to  and  support 
peers,  affiliation,  amicability,  concern  for  Soldier  quality  of  life,  conflict  management,  dependability,  and 
team  orientation)  will  allow  us  to  assess  the  degree  to  which  the  markers  are  observed  in  an  individual’s 
behavior.  The  methods,  which  are  most  appropriate  for  assessing  this  group  of  KSAs,  require  the 
identification  of  appropriate  scenarios/tasks  and  the  markers  of  the  construct  Build  and  Maintain 
Relationships.  For  Soldiers  who  have  experience  in  the  military,  performance  ratings  should  also  reveal 
behaviors  related  to  this  KSA.  Biodata  or  historical  information  may  also  be  useful,  and  such  information 
can  be  gathered  using  a  questionnaire-type  form  or  through  the  use  of  structured  interviews. 
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Potential  problems  with  the  method:  This  trait  could  be  faked,  even  when  a  realistic  assessment  such  as  a 
Live  Action  Group  is  employed.  Most  Soldiers  will  understand  what  behaviors  are  expected  of  them. 
Detecting  faking  may  be  difficult,  and  scoring  is  difficult  and  time  consuming  and  requires  a  trained 
scorer  (ideally  two,  so  the  reliability  of  the  scoring  can  be  assessed).  With  scenario  based  fixed  response 
“tests,”  creating  and  validating  the  scenarios  and  a  reasonable  range  of  markers  is  a  challenging  but 
doable  task.  Questionnaires  or  structured  interviews  intended  to  gather  this  information  might  not  be 
useful  for  Soldiers  with  little  or  no  relevant  work  experience 


Ability  to  Relate  to  and  Support  Peers.  The  degree  to  which  the  individual  treats  peers  in  a  courteous, 
respectful,  and  tactful  manner.  Provides  help  and  assistance  to  others.  Backs  up  and  fills  in  for  others  * 
when  needed.  Works  effectively  as  a  team  member. 

Assessment  methods  in  order  of  UA: 

( 1 )  Real  life  behavior  UA  =  9- 1 8 

(2) Live  Action  UA  =  10-12 

(3)  Simulations  UA=  10-12 

(4)  Interview  UA  =  6-7 

(5)  Self  report  UA  =  6-7 

Affiliation.  The  degree  of  sociability  that  one  exhibits.  Is  outgoing,  participative,  and  friendly  versus  shy 
and  reserved. 

Assessment  methods  in  order  of  UA: 

( 1 )  Li ve  action  UA  =  1 0- 1 2 

(2)  Simulations  UA  =  10-12 

(3)  Real  life  behavior  UA  =  8 

(4)  Self  report  UA  =  6-7 

(5)  Interviews  UA  =  5 


Amicability.  The  degree  of  pleasantness  versus  unpleasantness  exhibited  in  interpersonal  relations. 
Exhibits  good  will  towards  others  and  an  absence  of  antagonism.  Is  tactful  and  helpful  rather  than 
defensive,  touchy,  and  generally  contrary. 

Assessment  methods  in  order  of  UA: 


(l)Live  action 

UA=  10 

(2)  Simulations 

UA  =  8-10  • 

(3)  Real  life  behavior 

UA  =  7 

(4)  Interviews 

UA  =  7 

(5)  Self  report 

UA  =  6-7 

Concern  for  Soldier  Quality  of  Life. 

Is  sensitive  to  others’ 

assist  them  in  making  their  personal  and  family  life  better. 

Assessment  methods  in  order  of  UA: 

(l)Live  action 

UA  =  10-12 

(2)  Self-report 

UA  =  5-8 

(3)  Real  life  behavior 

UA  =  7 

(4)  Self-report 

UA  =  6-8 

(5)  Simulations 

UA  =  4-5 
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Conflict  Management.  The  degree  to  which  the  individual  encourages  and  supports  different 
perspectives,  avoids  harmful  conflict,  constructively  addresses  disagreements  that  undermine  team 
performance,  and  does  not  allow  conflicts  with  others  in  ways  that  preserve  good  relations  and  enhance 
trust. 


Assessment  methods  in  order  of  UA: 

(1)  Simulation 

(2)  Live  action 

(3)  Real  life  behavior 

(4)  Self-report 

(5)  Interviews 


UA=  10-12 
UA  =  8-12 
UA  =  7-8 
UA  =  5-7 
UA  =  4-5 


Cultural  Tolerance.  The  degree  to  which  an  individual  demonstrates  tolerance  and  understanding  of 
individuals  from  other  cultural  and  social  backgrounds,  both  in  the  context  of  the  diversity  of  U.S.  Army 
personnel  and  interactions  with  foreign  nationals  during  deployments  or  when  training  for  deployment. 

Assessment  methods  in  order  of  UA: 


( 1 )  Interviews  UA  =  8-12 

(2)  Live  action  UA  =  8 

(3)  Real  life  behavior  UA  =  7-8 

(4)  Simulation  UA  =  6-8 

(5)  Self-report  UA  =  5-8 


Dependability.  The  person’s  characteristic  degree  of  conscientiousness.  Is  disciplined,  well  organized, 
planful,  and  respectful  of  laws  and  regulations. 

Assessment  methods  in  order  of  UA: 


( 1 )  Real  life  behavior  UA  =16 

(2)  Interviews  UA  =  5 

(3)  Live  action  UA  =  4-5 

(4)  Simulation  UA  =  2-3 

(5)  Self-report  UA  =  0-3 


Team  Orientation.  The  degree  to  which  an  individual  identifies  with  the  team  and  other  team  members 
and  works  to  boost  team  morale  and  increase  the  team  bond  by  creating  and  maintaining  a  supportive 
work  environment;  willingness  to  put  the  needs  of  the  team  ahead  of  personal  needs. 

Assessment  methods  in  order  of  UA: 

( 1 )  Interviews  UA  =10-12 

(2)  Live  action  UA  =10 

(3)  Self-report  UA  =  7- 1 0 

(4)  Real  life  behavior  UA  =  7-8 

(5)  Simulation  UA  =  6-8 


Group  2:  Social  Intelligence  Construct. 

Rationale  for  method  selection:  These  KSAs  assess  the  Soldier’s  social  judgments,  flexibility,  and  ability 
to  work  in  harmony  with  others.  Asking  Soldiers  to  perform  a  task  in  a  simulation  or  role  play  that 
requires  that  the  Soldier  demonstrate  the  behaviors  that  are  markers  for  the  Social  Intelligence  Construct 
will  allow  us  to  assess  the  degree  to  which  the  markers  are  observed  in  an  individual’s  behavior.  The 
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methods,  which  are  most  appropriate  for  assessing  this  group  of  KSAs,  require  the  identification  of 
appropriate  scenarios/tasks  and  the  markers  of  the  construct  Build  and  Maintain  Relationships. 

Potential  problems  with  the  method:  This  trait  could  be  “faked,”  even  when  a  realistic  assessment  such  as 
a  Live  Action  Group  is  employed.  Most  Soldiers  will  understand  what  behaviors  are  expected  of  them. 
Detecting  faking  may  be  more  difficult  in  Live  Action  Group  exercises,  and  scoring  is  difficult  and  time 
consuming  and  requires  a  trained  scorer  (usually  two  to  assess  the  reliability  of  the  assessment).  With 
scenario-based  fixed-  response  tests  creating  and  validating  the  scenarios  and  a  reasonable  range  of 
markers  is  a  challenging  but  doable  task. 

Adaptability/Flexibility.  The  degree  to  which  an  individual  is  able  to  respond  to  rapidly  changing 
situations  (e.g.,  assignments,  relocation,  new  Soldiers)  and  accept  new  roles. 

Assessment  methods  in  order  of  UA: 

(1)  Interviews  UA  =  10 

(2)  Live  action  UA=  10-12 

(3)  Simulation  UA  =  8-10 

(4)  Self-report  UA  =  5-14 

(5)  Real  life  behavior  UA  =  7-8 

Coordination.  The  ability  to  work  interdependently  to  reach  task  completion,  share  information  and 
effort,  and  work  together  with  others.  Can  adjust  own  time  and  work  activities  to  ensure  interdependent 
tasks  are  completed  effectively. 

Assessment  methods  in  order  of  UA: 


(l)Live  action 

UA  =  10-14 

(2)  Simulation 

UA=  10-12 

(3)  Real  life  behavior 

UA  =  7-8 

(4)  Self-report 

UA  =  4-8 

(5)  Interviews 

UA  =  0-6 

Social  Perceptiveness.  The  degree  to  which  an  individual  is  able  to  monitor  own  and  other’s  emotions, 
discriminate  among  them,  and  use  the  information  to  guide  one’s  thinking  and  actions,  allowing  one  to 
work  cooperatively  with  others.  Is  aware  of  how  own  behavior  impacts  others. 


Assessment  methods  in  order  of  UA: 

(1)  Simulation 

(2)  Live  action 

(3)  Interviews 

(4)  Self-report 

(5)  Real  life  behavior 


UA=  10-12 
UA  =  8-12 
UA  =  6-7 
UA  =  4-7 
UA  =  0-7 


Group  3:  Communications  Construct 

Rationale  for  method  selection:  These  KSAs  assess  the  Soldier’s  ability  to  communicate  effectively,  both 
non-verbally  and  through  verbal  means  (written  and  spoken).  Asking  Soldiers  to  perform  a  task  in  a 
simulation  or  role  play  that  requires  that  the  Soldier  demonstrate  the  behaviors  that  are  markers  for  the 
Communication  Construct  will  allow  us  to  assess  the  degree  to  which  the  non-verbal  and  spoken  markers 
are  observed  in  an  individual’s  behavior.  The  non-verbal  and  oral  communication  KSAs  can  be  assessed 
during  live  action  assessment  techniques.  The  KSA  for  written  communication  can  be  best  assessed 
through  writing  samples  or  from  performance  ratings  and  work  product  reviews.  The  methods,  which  are 
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most  appropriate  for  assessing  this  group  of  KSAs,  require  the  identification  of  appropriate  nonverbal  and 
verbal  communication  tasks  and  the  markers  of  the  construct  Communication. 

Potential  problems  with  the  method:  This  trait  will  be  difficult  to  fake  and  can  be  objectively  scored. 
Scoring  of  active  listening,  nonverbal  and  verbal  communication  behaviors  may  be  more  difficult  and 
time  consuming  than  scoring  written  communications,  and  requires  a  trained  scorer  (usually  two  to  assess 
the  reliability  of  the  assessment). 

Active  Listening.  The  ability  to  listen  non-evaluatively  to  instructions  and  related  messages,  focusing  on 
the  person  communicating.  Asks  and  responds  to  questions  as  appropriate.  Pays  attention  to  nonverbal 
cures  to  help  clarify/interpret  messages. 

Assessment  methods  in  order  of  UA: 

( 1 )  Li  ve  action  UA  =  1 2- 1 6 

(2)  Interviews  UA  =  1 2- 1 4 

(3)  Simulation  UA  =  8 

(4)  Real  life  behavior  UA  =  0-6 

(5)  Self-report  UA  =  0 

Nonverbal  Skills.  The  ability  to  match  verbal  and  nonverbal  messages  when  speaking  to  clarify  and 
enhance  the  message.  The  degree  to  which  an  individual  can  accurately  interpret  the  nonverbal  signals  of 
others. 

Assessment  methods  in  order  of  UA: 

( 1 )  Interviews  UA  =16 

(2)  Live  action  UA  =12-14 

(3)  Simulation  UA  =  0 

(4)  Self-report  UA  =  0 

(5)  Real  life  behavior  UA  =  0 

Oral  Communication  Skills.  The  ability  to  speak  clearly  and  precisely  so  that  others  can  easily 
understand.  The  ability  to  adapt  speaking  style  and  comments  to  the  audience  as  appropriate. 

Assessment  methods  in  order  of  UA: 


( 1 )  Interviews  UA  =  1 6 

(2)  Live  action  UA  =16 

(3)  Real  life  behavior  UA  =  0-8 

(4)  Simulation  UA  =  0-7 

(5)  Self-report  UA  =  0 


Written  Communication  Skills.  The  ability  to  write  clearly  so  that  message  is  understood  by  the  reader. 

Is  sensitive  to  the  limitations  of  written  communication  and  carefully  phrases  message  (e.g.,  email)  so  that 
the  intent  is  clearly  understood  by  the  receiver. 

Assessment  methods: 

(1)  Real  life  behavior  UA  =  8-10 

Group  4:  Team  Membership  Construct 

Ability  to  Provide  Feedback  to  Team  Members.  Provides  constructive  suggestions  to  other  team 
members  for  improving  their  own  and  team  performance.  Informs  team  members  when  their  performance 
is  hindering  achievement  of  team  mission. 
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Assessment  methods  in  order  of  UA: 


( 1 )  Li  ve  action  UA  =  1 2 

(2)  Real  life  behavior  UA  =  6-7 

(3)  Interviews  UA  =  6 

(4)  Simulation  UA  =  0 

(5)  Self-report  UA  =  0 


Acts  as  a  Role  Model.  Exhibits  self-confidence  and  a  positive  attitude.  Presents  a  positive  and 
professional  image  of  self  and  the  Army  even  when  off  duty. 

Assessment  methods  in  order  of  UA: 


( 1 )  Real  life  behavior  UA  =  6- 1 6 

(2)  Live  action  UA  =12 

(3)  Interviews  UA  =  6 

(4)  Simulation  UA  =  0 

(5)  Self-report  UA  =  0 


Cooperativeness  in  Problem-Solving :  The  ability  to  take  advantage  of  multiple  perspectives  to  find 
effective  solutions  to  problems. 

Assessment  methods  in  order  of  UA: 


(l)Live  action 

UA  =  8-10 

(2)  Real  life  behavior 

UA  =  7-8 

(3)  Interviews 

UA  =  4 

(4)  Self-report 

UA  =  0-6 

(5)  Simulation 

UA  =  0-3 

Helping  Others.  When  working  with  others,  helps  them  to  identify  individual  performance  strengths  and 
weaknesses;  helps  them  to  improve  performance.  Willingness  to  provide  assistance  as  needed  and  to 
guides  and  tutor  others  on  technical  matters. 


Assessment  methods  in  order  of  UA: 

(1)  Simulation 

(2)  Live  action 

(3)  Self-report 

(4)  Interviews 

(5)  Real  life  behavior 


UA  =  8-10 
UA  =  6-10 
UA  =  4-7 
UA  =  5-6 
UA  =  0-6 


Task  Leadership.  Ability  to  help  keep  the  team  focused  on  the  team’s  assignment  or  mission,  working 
with  team  members  to  react  to  changes  and  to  ensure  that  conflicts  do  not  hinder  mission  achievement. 

Assessment  methods  in  order  of  UA: 


(l)Live  action 

UA  =  14-16 

(2)  Real  life  behavior 

UA  =  7 

(3)  Simulation 

UA  =  5-6 

(4)  Self-report 

UA  =  5-6 

(5)  Interviews 

UA  =  5 
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